The status quo
Back in 2015, I bought an off-the-shelf NAS, a QNAP TS-453mini, to act as my file store and Plex server. I had previously owned a Synology box, and whilst I liked the Synology OS and experience, the hardware was underwhelming. I loaded up the successor QNAP with four 5TB drives in RAID10, and moved all my files over (after some initial DoA drive issues were handled).
QNAP TS-453mini product photo
That thing has been in service for about 8 years now, and it s been a mixed bag. It was definitely more powerful than the predecessor system, but it was clear that QNAP s OS was not up to the same standard as Synology s perhaps best exemplified by HappyGet 2 , the QNAP webapp for downloading videos from streaming services like YouTube, whose icon is a straight rip-off of StarCraft 2. On its own, meaningless but a bad omen for overall software quality
The logo for QNAP HappyGet 2 and Blizzard s StarCraft 2 side by side
Additionally, the embedded Celeron processor in the NAS turned out to be an issue for some cases. It turns out, when playing back videos with subtitles, most Plex clients do not support subtitles properly instead they rely on the Plex server doing JIT transcoding to bake the subtitles directly into the video stream. I discovered this with some Blu-Ray rips of Game of Thrones some episodes would play back fine on my smart TV, but episodes with subtitled Dothraki speech would play at only 2 or 3 frames per second.
The final straw was a ransomware attack, which went through all my data and locked every file below a 60MiB threshold. Practically all my music gone. A substantial collection of downloaded files, all gone. Some of these files had been carried around since my college days digital rarities, or at least digital detritus I felt a real sense of loss at having to replace. This episode was caused by a ransomware targeting specific vulnerabilities in the QNAP OS, not an error on my part.
So, I decided to start planning a replacement with:
A non-garbage OS, whilst still being a NAS-appliance type offering (not an off-the-shelf Linux server distro)
Full remote management capabilities
A small form factor comparable to off-the-shelf NAS
A powerful modern CPU capable of transcoding high resolution video
All flash storage, no spinning rust
At the time, no consumer NAS offered everything (The Asustor FS6712X exists now, but didn t when this project started), so I opted to go for a full DIY rather than an appliance not the first time I ve jumped between appliances and DIY for home storage.
Selecting the core of the system
There aren t many companies which will sell you a small motherboard with IPMI. Supermicro is a bust, so is Tyan. But ASRock Rack, the server division of third-tier motherboard vendor ASRock, delivers. Most of their boards aren t actually compliant Mini-ITX size, they re a proprietary Deep Mini-ITX with the regular screw holes, but 40mm of extra length (and a commensurately small list of compatible cases). But, thankfully, they do have a tiny selection of boards without the extra size, and I stumbled onto the X570D4I-2T, a board with an AMD AM4 socket and the mature X570 chipset. This board can use any AMD Ryzen chip (before the latest-gen Ryzen 7000 series); has built in dual 10 gigabit ethernet; IPMI; four (laptop-sized) RAM slots with full ECC support; one M.2 slot for NVMe SSD storage; a PCIe 16x slot (generally for graphics cards, but we live in a world of possibilities); and up to 8 SATA drives OR a couple more NVMe SSDs. It s astonishingly well featured, just a shame it costs about $450 compared to a good consumer-grade Mini ITX AM4 board costing less than half that.
I was so impressed with the offering, in fact, that I crowed about it on Mastodon and ended up securing ASRock another sale, with someone else looking into a very similar project to mine around the same timespan.
The next question was the CPU. An important feature of a system expected to run 24/7 is low power, and AM4 chips can consume as much as 130W under load, out of the box. At the other end, some models can require as little as 35W under load the OEM-only GE suffix chips, which are readily found for import on eBay. In their PRO variant, they also support ECC (all non-G Ryzen chips support ECC, but only Pro G chips do). The top of the range 8 core Ryzen 7 PRO 5750GE is prohibitively expensive, but the slightly weaker 6 core Ryzen 5 PRO 5650GE was affordable, and one arrived quickly from Hong Kong. Supplemented with a couple of cheap 16 GiB SODIMM sticks of DDR4 PC-3200 direct from Micron for under $50 a piece, that left only cooling as an unsolved problem to get a bootable test system.
The official support list for the X570D4I-2T only includes two rackmount coolers, both expensive and hard to source. The reason for such a small list is the non standard cooling layout of the board instead of an AM4 hole pattern with the standard plastic AM4 retaining clips, it has an Intel 115x hole pattern with a non-standard backplate (Intel 115x boards have no backplate, the stock Intel 115x cooler attaches to the holes with push pins). As such every single cooler compatibility list excludes this motherboard. However, the backplate is only secured with a mild glue with minimal pressure and a plastic prying tool it can be removed, giving compatibility with any 115x cooler (which is basically any CPU cooler for more than a decade). I picked an oversized low profile Thermalright AXP120-X67 hoping that its 120mm fan would cool the nearby MOSFETs and X570 chipset too.
Thermalright AXP120-X67, AMD Ryzen 5 PRO 5650GE, ASRock Rack X570D4I-2T, all assembled and running on a flat surface
Testing up to this point
Using a spare ATX power supply, I had enough of a system built to explore the IPMI and UEFI instances, and run MemTest86 to validate my progress. The memory test ran without a hitch and confirmed the ECC was working, although it also showed that the memory was only running at 2933 MT/s instead of the rated 3200 MT/s (a limit imposed by the motherboard, as higher speeds are considered overclocking). The IPMI interface isn t the best I ve ever used by a long shot, but it s minimum viable and allowed me to configure the basics and boot from media entirely via a Web browser.
Memtest86 showing test progress, taken from IPMI remote control window
One sad discovery, however, which I ve never seen documented before, on PCIe bifurcation.
With PCI Express, you have a number of lanes which are allocated in groups by the motherboard and CPU manufacturer. For Ryzen prior to Ryzen 7000, that s 16 lanes in one slot for the graphics card; 4 lanes in one M.2 connector for an SSD; then 4 lanes connecting the CPU to the chipset, which can offer whatever it likes for peripherals or extra lanes (bottlenecked by that shared 4x link to the CPU, if it comes down to it).
It s possible, with motherboard and CPU support, to split PCIe groups up for example an 8x slot could be split into two 4x slots (eg allowing two NVMe drives in an adapter card NVME drives these days all use 4x). However with a Cezanne Ryzen with integrated graphics, the 16x graphics card slot cannot be split into four 4x slots (ie used for for NVMe drives) the most bifurcation it allows is 8x4x4x, which is useless in a NAS.
Screenshot of PCIe 16x slot bifurcation options in UEFI settings, taken from IPMI remote control window
As such, I had to abandon any ideas of an all-NVMe NAS I was considering: the 16x slot split into four 4x, combined with two 4x connectors fed by the X570 chipset, to a total of 6 NVMe drives. 7.6TB U.2 enterprise disks are remarkably affordable (cheaper than consumer SATA 8TB drives), but alas, I was locked out by my 5650GE. Thankfully I found out before spending hundreds on a U.2 hot swap bay. The NVMe setup would be nearly 10x as fast as SATA SSDs, but at least the SATA SSD route would still outperform any spinning rust choice on the market (including the fastest 10K RPM SAS drives)
Containing the core
The next step was to pick a case and power supply. A lot of NAS cases require an SFX (rather than ATX) size supply, so I ordered a modular SX500 unit from Silverstone. Even if I ended up with a case requiring ATX, it s easy to turn an SFX power supply into ATX, and the worst result is you have less space taken up in your case, hardly the worst problem to have.
That said, on to picking a case. There s only one brand with any cachet making ITX NAS cases, Silverstone. They have three choices in an appropriate size: CS01-HS, CS280, and DS380. The problem is, these cases are all badly designed garbage. Take the CS280 as an example, the case with the most space for a CPU cooler. Here s how close together the hotswap bay (right) and power supply (left) are:
Internal image of Silverstone CS280 NAS build. Image stolen from ServeTheHome
With actual cables connected, the cable clearance problem is even worse:
Internal image of Silverstone CS280 NAS build. Image stolen from ServeTheHome
Remember, this is the best of the three cases for internal layout, the one with the least restriction on CPU cooler height. And it s garbage! Total hot garbage! I decided therefore to completely skip the NAS case market, and instead purchase a 5.25 -to-2.5 hot swap bay adapter from Icy Dock, and put it in an ITX gamer case with a 5.25 bay. This is no longer a served market 5.25 bays are extinct since nobody uses CD/DVD drives anymore. The ones on the market are really new old stock from 2014-2017: The Fractal Design Core 500, Cooler Master Elite 130, and Silverstone SUGO 14. Of the three, the Fractal is the best rated so I opted to get that one however it seems the global supply of new old stock fully dried up in the two weeks between me making a decision and placing an order leaving only the Silverstone case.
Icy Dock have a selection of 8-bay 2.5 SATA 5.25 hot swap chassis choices in their ToughArmor MB998 series. I opted for the ToughArmor MB998IP-B, to reduce cable clutter it requires only two SFF-8611-to-SF-8643 cables from the motherboard to serve all eight bays, which should make airflow less of a mess. The X570D4I-2T doesn t have any SATA ports on board, instead it has two SFF-8611 OCuLink ports, each supporting 4 PCI Express lanes OR 4 SATA connectors via a breakout cable. I had hoped to get the ToughArmor MB118VP-B and run six U.2 drives, but as I said, the PCIe bifurcation issue with Ryzen G chips meant I wouldn t be able to run all six bays successfully.
NAS build in Silverstone SUGO 14, mid build, panels removedSilverstone SUGO 14 from the front, with hot swap bay installed
Actual storage for the storage server
My concept for the system always involved a fast boot/cache drive in the motherboard s M.2 slot, non-redundant (just backups of the config if the worst were to happen) and separate storage drives somewhere between 3.8 and 8 TB each (somewhere from $200-$350). As a boot drive, I selected the Intel Optane SSD P1600X 58G, available for under $35 and rated for 228 years between failures (or 11,000 complete drive rewrite cycles).
So, on to the big expensive choice: storage drives. I narrowed it down to two contenders: new-old-stock Intel D3-S4510 3.84TB enterprise drives, at about $200, or Samsung 870 QVO 8TB consumer drives, at about $375. I did spend a long time agonizing over the specification differences, the ZFS usage reports, the expected lifetime endurance figures, but in reality, it came down to price $1600 of expensive drives vs $3200 of even more expensive drives. That s 27TB of usable capacity in RAID-Z1, or 23TB in RAID-Z2. For comparison, I m using about 5TB of the old NAS, so that s a LOT of overhead for expansion.
Storage SSD loaded into hot swap sled
Booting up
Bringing it all together is the OS. I wanted an appliance NAS OS rather than self-administering a Linux distribution, and after looking into the surrounding ecosystems, decided on TrueNAS Scale (the beta of the 2023 release, based on Debian 12).
TrueNAS Dashboard screenshot in browser window
I set up RAID-Z1, and with zero tuning (other than enabling auto-TRIM), got the following performance numbers:
Finally, the numbers reported on the old NAS with four 7200 RPM hard disks in RAID 10:
IOPS
Bandwidth
4k random writes
430
1.7 MiB/s
4k random reads
8006
32 MiB/s
Sequential writes
311 MiB/s
Sequential reads
566 MiB/s
Performance seems pretty OK. There s always going to be an overhead to RAID. I ll settle for the 45x improvement on random writes vs. its predecessor, and 4.5x improvement on random reads. The sequential write numbers are gonna be impacted by the size of the ZFS cache (50% of RAM, so 16 GiB), but the rest should be a reasonable indication of true performance.
It took me a little while to fully understand the TrueNAS permissions model, but I finally got Plex configured to access data from the same place as my SMB shares, which have anonymous read-only access or authenticated write access for myself and my wife, working fine via both Linux and Windows.
And that s it! I built a NAS. I intend to add some fans and more RAM, but that s the build. Total spent: about $3000, which sounds like an unreasonable amount, but it s actually less than a comparable Synology DiskStation DS1823xs+ which has 4 cores instead of 6, first-generation AMD Zen instead of Zen 3, 8 GiB RAM instead of 32 GiB, no hardware-accelerated video transcoding, etc. And it would have been a whole lot less fun!
The final system, powered up
(Also posted on PCPartPicker)
DebConf23, the 24th annual Debian Developer Conference,
is taking place in Kochi, India from September 10th to 17th, 2023.
Debian contributors from all over the world have come together at Infopark, Kochi
to participate and work in a conference exclusively run by volunteers.
Today the main conference starts with over 373 expected attendants and 92
scheduled activities, including 45-minute and 20-minute talks, Bird of a Feather
("BoF") team meetings, workshops, a job fair, as well as a variety of other
events.
The full schedule is updated each
day, including activities planned ad-hoc by attendees over the course of the
conference.
If you would like to engage remotely, you can follow the video streams
available from the DebConf23 website for the events happening in the three talk
rooms: Anamudi, Kuthiran and Ponmudi. Or you can join the conversations
happening inside the talk rooms via the OFTC IRC network
in the #debconf-anamudi, #debconf-kuthiran, and the #debconf-ponmudi
channels. Please also join us in the #debconf channel for common discussions
related to DebConf.
You can also follow the live coverage of news about DebConf23 provided by our
micronews service or the @debian profile on your
favorite social network.
DebConf is committed to a safe and welcoming environment for all participants.
Please see our Code of Conduct page on the DebConf23 website for more
information on this.
Debian thanks the commitment of numerous sponsors to support DebConf23,
particularly our Platinum Sponsors: Infomaniak, Proxmox and Siemens.
~
DebConf23, the 24th edition of the Debian
conference is taking place in Infopark at Kochi, Kerala, India.
Thanks to the hard work of its organizers, it will be, this year as well, an
interesting and fruitful event for attendees.
We would like to warmly welcome the sponsors of DebConf23, and
introduce them to you.
We have three Platinum sponsors.
Our first Platinum sponsor is Infomaniak.
Infomaniak is a key player in the European cloud market and the leading
developer of Web technologies in Switzerland. It aims to be an independent
European alternative to the web giants and is committed to an ethical and
sustainable Web that respects privacy and creates local jobs. Infomaniak
develops cloud solutions (IaaS, PaaS, VPS), productivity tools for online
collaboration and video and radio streaming services.
Proxmox is our second Platinum sponsor.
Proxmox develops powerful, yet easy-to-use open-source server software.
The product portfolio from Proxmox, including server virtualization, backup,
and email security, helps companies of any size, sector, or industry to
simplify their IT infrastructures. The Proxmox solutions are based on the
great Debian platform, and we are happy that we can give back to the
community by sponsoring DebConf23.
Siemens is our third Platinum sponsor.
Siemens is a technology company focused on industry, infrastructure and
transport. From resource-efficient factories, resilient supply
chains, smarter buildings and grids, to cleaner and more comfortable
transportation, and advanced healthcare, the company creates
technology with purpose adding real value for customers. By combining
the real and the digital worlds, Siemens empowers its customers to
transform their industries and markets, helping them to enhance the
everyday of billions of people.
Our Gold sponsors are:
Lenovo,
Lenovo is a global technology leader manufacturing a wide portfolio of
connected products including smartphones, tablets, PCs and workstations
as well as AR/VR devices, smart home/office and data center solutions.
Freexian,
Freexian is a services company specialized in Free Software and in
particular Debian GNU/Linux, covering consulting, custom developments,
support, training. Freexian has a recognized Debian expertise thanks to the
participation of Debian developers.
Google,
Google is one of the largest technology companies in the world, providing a
wide range of Internet-related services and products such as online
advertising technologies, search, cloud computing, software, and hardware.
Ubuntu,
the Operating System delivered by Canonical.
Collabora, a global consultancy delivering
Open Source software solutions to the commercial world.
FOSS United,
a non-profit foundation that aims at promoting and strengthening the Free
and Open Source Software (FOSS) ecosystem in India.
The FSIJ, a non-profit organization
dedicated to supporting Free Software growth and development.
Civil Infrastructure Platform,
a collaborative project hosted by the Linux Foundation,
establishing an open source base layer of industrial grade software.
Roche, a major international pharmaceutical
provider and research company dedicated to personalized healthcare.
Two Sigma,
rigorous inquiry, data analysis, and invention to help solve the toughest
challenges across financial services.
Matanel Foundation, operates in Israel,
as its first concern is to preserve the cohesion of a society and a nation
plagued by divisions.
The FOSSEE (Free/Libre and Open Source Software for
Education) project promotes the use of FLOSS tools in academia and research.
The project is part of the National Mission on Education through Information
and Communication Technology (ICT), Ministry of Education (MoE), Government of
India.
Arm: with the world s Best SoC Design Portfolio,
Arm powered solutions have been supporting innovation for more than 30 years
and are deployed in over 225 billion chips to date.
Kerala Vision Broadband is a
subsidiary of Kerala State IT Infrastructure Limited (KSITIL) which provides
high-speed internet services in the state of Kerala, India.
A special thanks to the Infoparks Kerala,
our Venue Partner!
Thanks to all our sponsors for their support!
Their contributions make it possible for a large number
of Debian contributors from all over the globe to work together,
help and learn from each other in DebConf23.
Another busy week in the KDE snap world. Most of the release-service apps are in candidate channel waiting to be tested. Testing is the bottle neck in the process, so I am trying something new and calling for help! Please test your favorite apps and report on https://discuss.kde.org/t/all-things-snaps-questions-concerns-praise/ any issues and which apps tested. Thanks!
There are some very big fixes in this release:
Desktop file defined so xdg-desktop-portals will now work.
Print support in many apps where it made sense. Please let me know if I missed one.
The KF6 content pack is coming along nicely using qt-framework-sdk snap!
Qt5 content snap using KDE patch set is nearly complete!
I believe I have a solution for our PIM applications by creating an Akondai dbus provider snap and setting all the PIM applications as consumers. I am waiting for manual review to pass.
I have a pile of new applications waiting for reserved name approvals. Igor has pinged the relevant people to speed this normally quick process up.
The pushback on per repository snapcraft files has stopped, so I have begun the process, which will take some time. This is a huge step in automating snap builds and cutting down my manual work so I can do more exciting things like plasma snaps!
Some big news on my project, a big thank you goes out to Kevin Ottens for reaching out, his company does exactly what I need to move it forward. I will update as we hash out the details, but it looks like my project isn t dead after all!
I know many have asked Why haven t you given up already?? The answer in short, I am stubborn. I refuse to give up on anything until I am given a good reason to. When I started my path in computers oh so many years ago, you would be surprised how many people told me to give it up, you ll never make it as a woman. Challenge accepted. Here I am, still going strong. When I want something, I go get it, no matter what it takes!
I still need to have a ( somewhat desperate ) call for donations. This will hopefully end soon, but for now, please consider donating to my September survival fund! Please share with anyone you that may find my work useful in any way. Thanks for your consideration
PS: Debian uploads for bubble-gum are moving along. Please if you have any packaging you need done in Debian proper, let me know and I will get on it, time allowing of course.
There is a bit of context that needs to be shared before I get to this and would be a long one. For reasons known and unknown, I have a lot of sudden electricity outages. Not just me, all those who are on my line. A discussion with a lineman revealed that around 200+ families and businesses are on the same line and when for whatever reason the electricity goes for all. Even some of the traffic lights don t work. This affects software more than hardware or in some cases, both. And more specifically HDD s are vulnerable. I had bought an APC unit several years for precisely this, but over period of time it just couldn t function and trips also when the electricity goes out. It s been 6-7 years so can t even ask customer service to fix the issue and from whatever discussions I have had with APC personnel, the only meaningful difference is to buy a new unit but even then not sure this is an issue that can be resolved, even with that.
That comes to the issue that happens once in a while where the system fsck is unable to repair /home and you need to use an external pen drive for the same. This is my how my hdd stacks up / is on dev/sda7 /boot is on /dev/sda6, /boot/efi is on /dev/sda2 and /home is on /dev/sda8 so theoretically, if /home for some reason doesn t work I should be able drop down on /dev/sda7, unmount /dev/sda8, run fsck and carry on with my work. I tried it number of times but it didn t work. I was dropping down on tty1 and attempting the same, no dice as root/superuser getting the barest x-term. So first I tried asking couple of friends who live nearby me. Unfortunately, both are MS-Windows users and both use what are called as company-owned laptops . Surfing on those systems were a nightmare. Especially the number of pop-ups of ads that the web has become. And to think about how much harassment ublock origin has saved me over the years. One of the more interesting bits from both their devices were showing all and any downloads from fosshub was showing up as malware. I dunno how much of that is true or not as haven t had to use it as most software we get through debian archives or if needed, download from github or wherever and run/install it and you are in business. Some of them even get compiled into a good .deb package but that s outside the conversation atm. My only experience with fosshub was few years before the pandemic and that was good. I dunno if fosshub really has malware or malwarebytes was giving false positives. It also isn t easy to upload a 600 MB+ ISO file somewhere to see whether it really has malware or not. I used to know of a site or two where you could upload a suspicious file and almost 20-30 famous and known antivirus and anti-malware engines would check it and tell you the result. Unfortunately, I have forgotten the URL and seeing things from MS-Windows perspective, things have gotten way worse than before.
So left with no choice, I turned to the local LUG for help. Fortunately, my mobile does have e-mail and I could use gmail to solicit help. While there could have been any number of live CD s that could have helped but one of my first experiences with GNU/Linux was that of Knoppix that I had got from Linux For You (now known as OSFY) sometime in 2003. IIRC, had read an interview of Mr. Klaus Knopper as well and was impressed by it. In those days, Debian wasn t accessible to non-technical users then and Knoppix was a good tool to see it. In fact, think he was the first to come up with the idea of a Live CD and run with it while Canonical/Ubuntu took another 2 years to do it. I think both the CD and the interview by distrowatch was shared by LFY in those early days. Of course, later the story changes after he got married, but I think that is more about Adriane rather than Knoppix. So Vishal Rao helped me out. I got an HP USB 3.2 32GB Type C OTG Flash Drive x5600c (Grey & Black) from a local hardware dealer around similar price point. The dealer is a big one and has almost 200+ people scattered around the city doing channel sales who in turn sell to end users. Asking one of the representatives about their opinion on stopping electronic imports (apparently more things were added later to the list including all sorts of sundry items from digital cameras to shavers and whatnot.) The gentleman replied that he hopes that it would not happen otherwise more than 90% would have to leave their jobs. They already have started into lighting fixtures (LED bulbs, tubelights etc.) but even those would come in the same ban
The main argument as have shared before is that Indian Govt. thinks we need our home grown CPU and while I have no issues with that, as shared before except for RISC-V there is no other space where India could look into doing that. Especially after the Chip Act, Biden has made that any new fabs or any new thing in chip fabrication will only be shared with Five Eyes only. Also, while India is looking to generate about 2000 GW by 2030 by solar, China has an ambitious 20,000 GW generation capacity by the end of this year and the Chinese are the ones who are actually driving down the module prices. The Chinese are also automating their factories as if there s no tomorrow. The end result of both is that China will continue to be the world s factory floor for the foreseeable future and whoever may try whatever policies, it probably is gonna be difficult to compete with them on prices of electronic products. That s the reason the U.S. has been trying so that China doesn t get the latest technology but that perhaps is a story for another day.
HP USB 3.2 Type C OTG Flash Drive x5600c
For people who have had read this blog they know that most of the flash drives today are MLC Drives and do not have the longevity of the SLC Drives. For those who maybe are new, this short brochure/explainer from Kingston should enhance your understanding. SLC Drives are rare and expensive. There are also a huge number of counterfeit flash drives available in the market and almost all the companies efforts whether it s Kingston, HP or any other manufacturer, they have been like a drop in the bucket. Coming back to the topic at hand. While there are some tools that can help you to figure out whether a pen drive is genuine or not. While there are products that can tell you whether they are genuine or not (basically by probing the memory controller and the info. you get from that.) that probably is a discussion left for another day. It took me couple of days and finally I was able to find time to go Vishal s place. The journey of back and forth lasted almost 6 hours, with crazy traffic jams. Tells you why Pune or specifically the Swargate, Hadapsar patch really needs a Metro. While an in-principle nod has been given, it probably is more than 5-7 years or more before we actually have a functioning metro. Even the current route the Metro has was supposed to be done almost 5 years to the date and even the modified plan was of 3 years ago. And even now, most of the Stations still need a lot of work to be done. PMC, Deccan as examples etc. still have loads to be done. Even PMT (Pune Muncipal Transport) that that is supposed to do the last mile connections via its buses has been putting half-hearted attempts
Vishal Rao
While Vishal had apparently seen me and perhaps we had also interacted, this was my first memory of him although we have been on a few boards now and then including stackexchange. He was genuine and warm and shared 4-5 distros with me, including Knoppix and System Rescue as shared by Arun Khan. While this is and was the first time I had heard about Ventoy apparently Vishal has been using it for couple of years now. It s a simple shell script that you need to download and run on your pen drive and then just dump all the .iso images. The easiest way to explain ventoy is that it looks and feels like Grub. Which also reminds me an interaction I had with Vishal on mobile. While troubleshooting the issue, I was unsure whether it was filesystem that was the issue or also systemd was corrupted. Vishal reminded me of putting fastboot to the kernel parameters to see if I m able to boot without fscking and get into userspace i.e. /home. Although journalctl and systemctl were responding even on tty1 still was a bit apprehensive. Using fastboot was able to mount the whole thing and get into userspace and that told me that it s only some of the inodes that need clearing and there probably are some orphaned inodes. While Vishal had got a mini-pc he uses that a server, downloads stuff to it and then downloads stuff from it. From both privacy, backup etc. it is a better way to do things but then you need to laptop to access it. I am sure he probably uses it for virtualization and other ways as well but we just didn t have time for that discussion. Also a mini-pc can set you back anywhere from 25 to 40k depending on the mini-pc and the RAM and the SSD. And you need either a lappy or an Raspberry Pi with some kinda visual display to interact with the mini-pc. While he did share some of the things, there probably could have been a far longer interaction just on that but probably best left for another day.
Now at my end, the system I had bought is about 5-6 years old. At that time it only had 6 USB 2.0 drives and 2 USB 3.0 (A) drives.
The above image does tell of the various form factors. One of the other things is that I found the pendrive and its connectors to be extremely fiddly. It took me number of times fiddling around with it when I was finally able to put in and able to access the pen drive partitions. Unfortunately, was unable to see/use systemrescue but Knoppix booted up fine. I mounted the partitions briefly to see where is what and sure enough /dev/sda8 showed my /home files and folders. Unmounted it, then used $fsck -y /dev/sda8 and back in business.
This concludes what happened.
Updates Quite a bit was left out on the original post, part of which I didn t know and partly stuff which is interesting and perhaps need a blog post of their own. It s sad I won t be part of debconf otherwise who knows what else I would have come to know.
One of the interesting bits that I came to know about last week is the Alibaba T-Head T-Head TH1520 RISC-V CPU and saw it first being demoed on a laptop and then a standalone tablet. The laptop is an interesting proposition considering Alibaba opened up it s chip thing only couple of years ago. To have an SOC within 18 months and then under production for lappies and tablets is practically unheard of especially of a newbie/startup. Even AMD took 3-4 years for its first chip.It seems they (Alibaba) would be parceling them out by quarter end 2023 and another 1000 pieces/Units first quarter next year, while the scale is nothing compared to the behemoths, I think this would be more as a matter of getting feedback on both the hardware and software. The value proposition is much better than what most of us get, at least in India. For example, they are doing a warranty for 5 years and also giving spare parts. RISC-V has been having a lot of resurgence in China in part as its an open standard and partly development will be far cheaper and faster than trying x86 or x86-64. If you look into both the manufacturers, due to monopoly, both of them now give 5-8% increment per year, and if you look back in history, you would find that when more chips were in competition, they used to give 15-20% performance increment per year.
2. While Vishal did share with me what he used and the various ways he uses the mini-pc, I did have a fun speculating on what he could use it. As shared by Romane as his case has shared, the first thing to my mind was backups. Filesystems are notorious in the sense they can be corrupted or can be prone to be corrupted very easily as can be seen above . Backups certainly make a lot of sense, especially rsync.
The other thing that came to my mind was having some sort of A.I. and chat server. IIRC, somebody has put quite a bit of open source public domain data in debian servers that could be used to run either a chatbot or an A.I. or both and use that similar to how chatGPT but with much limited scope than what chatgpt uses. I was also thinking a media server which Vishal did share he does. I may probably visit him sometime to see what choices he did and what he learned in the process, if anything.
Another thing that could be done is just take a dump of any of commodity markets or any markets and have some sort of predictive A.I. or whatever. A whole bunch of people have scammed thousands of Indian users on this, but if you do it on your own and for your own purposes to aid you buy and sell stocks or whatever commodity you may fancy. After all, nowadays markets themselves are virtual.
While Vishal s mini-pc doesn t have any graphics, if it was an AMD APU mini-pc, something like this he could have hosted games in the way of thick server, thin client where all graphics processing happens on the server rather than the client. With virtual reality I think the case for the same case could be made or much more. The only problem with VR/AR is that we don t really have mass-market googles, eye pieces or headset. The only notable project that Google has/had in that place is the Google VR Cardboard headset and the experience is not that great or at least was not that great few years back when I could hear and experience the same. Most of the VR headsets say for example the Meta Quest 2 is for around INR 44k/- while Quest 3 is INR 50k+ and officially not available. As have shared before, the holy grail of VR would be when it falls below INR 10k/- so it becomes just another accessory, not something you really have to save for. There also isn t much content on that but then that is also the whole chicken or egg situation. This again is a non-stop discussion as so much has been happening in that space it needs its own blog post/article whatever.
Till later.
The meat has been fetched: those of us in the house are about to get bacon sandwiches. Pepper the dog is in the garden. Time for the mayhem to start, I think. Various folk are travelling here so it will soon be crowded: the weather is sunny but cool and it looks good for a three day weekend.
This is a huge effort that falls to Steve and Jo and a huge disruption for them each year - for which many thanks, as ever. [And, as is traditional on this blog, the posts only ever seem to appear from Cambridge].
This last weekend I completed a bike rides project I started during the first Covid lockdown in 2020:
I ve cycled to every settlement (and radio observatory) within 20km of my house, in alphabetical order.
Stir crazy
In early 2020, during the first lockdown, I was going a bit stir crazy. Clare said you re going very strange, you have to go out and get some exercise . After a bit of discussion, we came up with this plan: I d visit all the local villages, in alphabetical order.
Choosing the radius
I decided that I would pick a round number of kilometers, as the crow flies, from my house. 20km seemed about right. 25km would have included Ely, which would have been nice, but it would have added a great many places, all of them quite distant.
Software
I wrote a short Rust program to process OSM data into a list of places to visit, and their distances and bearings.
You can download a tarball of the alphabetical villages scanner. (I haven t published the git history because it has my house s GPS coordinates in it, and because I committed the output files from which that location can be derived.)
The Rides
I set off on my first ride, to Aldreth, on Sunday the 31st of May 2020. The final ride collected Yelling, on Saturday the 19th of August 2023.
I did quite a few rides in June and July 2020 - more than one a week. (I d read the lockdown rules, and although some of the government messaging said you should stay near your house, that wasn t in the legislation. Of course I didn t go into any buildings or anything.)
I m not much of a morning person, so I often set off after lunch. For the longer rides I would usually pack a picnic. Almost all of the rides I did just by myself. There were a handful where I had friends along:
Dry Drayton, which I collected with Clare, at night. I held my bike up so the light shone at the village sign, so we could take a photo of it.
Madingley, Melbourn and Meldreth, which was quite an expedition with my friend Ben. We went out as far as Royston and nearby Barley (both outside my radius and not on my list) mostly just so that my project would have visited Hertfordshire.
The Hemingfords, where I had my friend Matthew along, and we had a very nice pub lunch.
Girton and Wilburton, where I visited friends. Indeed, I stopped off in Wilburton on one or two other occasions.
And, of course, Yelling, for which there were four of us, again with a nice lunch (in Eltisley).
I had relatively little mechanical trouble. My worst ride for this was Exning: I got three punctures that day. Luckily the last one was close to home.
I often would stop to take lots of photos en-route. My mum in particular appreciated all the pretty pictures.
Rules
I decided on these rules:
I would cycle to each destination, in order, and it would count as collected if I rode both there and back. I allowed collecting multiple villages in the same outing, provided I did them in the right order. (And obviously I was allowed to pass through places out of order, without counting them.)
I tried to get a picture of the village sign, where there was one. Failing that, I got a picture of something in the village with the village s name on it. I think the only one I didn t manage this for was Westley Bottom; I had to make do with the word Westley on some railway level crossing equipment. In Barway I had to make do with a planning application, stuck to a pole.
I tried not to enter and leave a village by the same road, if possible.
Edge cases
I had to make some decisions:
I decided that I would consider the project complete if I visited everywhere whose centre was within my radius. But the centre of a settlement is rather hard to define. I needed a hard criterion for my OpenStreetMap data mining: a place counted if there was any node, way or relation, with the relevant place tag, any part of which was within my ambit. That included some places that probably oughtn t to have counted, but, fine.
I also decided that I wouldn t visit suburbs of Cambridge, separately from Cambridge itself. I don t consider them separate settlements, at least, not if they re conurbated with Cambridge. So that excluded Trumpington, for example. But I decided that Girton and Fen Ditton were (just) separable. Although the place where I consider Girton and Cambridge to nearly touch, is administratively well inside Girton, I chose to look at land use (on the ground, and in OSM data), rather than administrative boundaries.
But I did visit both Histon and Impington, and all each of the Shelfords and Stapleford, as separate entries in my list. Mostly because otherwise I d have to decide whether to skip (say) Impington, or Histon. Whereas skipping suburbs of Cambridge in favour of Cambridge itself was an easy decision, and it also got rid of a bunch of what would have been quite short, boring, urban expeditions.
I sorted all the Greats and Littles under G and L, rather than (say) Shelford, Great , which seemed like it would be cheating because then I would be able to do Shelford, Great and Shelford, Little in one go.
Northstowe turned from mostly a building site into something that was arguably a settlement, during my project. It wasn t included in the output of my original data mining. Of course it s conurbated with Oakington - but happily, Northstowe inserts right before Oakington in the alphabetical list, so I decided to add it, visiting both the old and new in the same day.
There are a bunch of other minor edge cases. Some villages have an outlying hamlet. Mostly I included these. There are some individual farms, which I generally didn t count.
Some stats
I visited 150 villages plus the Lords Bridge radio observatory. The project took 3 years and 3 months to complete.
There were 96 rides, totalling about 4900km. So my mean distance was around 51km. The median distance per ride was a little higher, at around 52 km, and the median duration (including stoppages) was about 2h40. The total duration, if you add them all up, including stoppages, was about 275h, giving a mean speed including photo stops, lunches and all, of 18kph.
The longest ride was 89.8km, collecting Scotland Farm, Shepreth, and Six Mile Bottom, so riding across the Cam valley. The shortest ride was 7.9km, collecting Cambridge (obviously); and I think that s the only one I did on my Brompton. The rest were all on my trusty Thorn Audax.
My fastest ride (ranking by distance divided by time spent in motion) was to collect Haddenham, where I covered 46.3km in 1h39, giving an average speed in motion of 28.0kph.
The most I collected in one day was 5 places: West Wickham, West Wratting, Westley Bottom, Westley Waterless, and Weston Colville. That was the day of the Wests. (There s only one East: East Hatley.)
Map
Here is a pretty picture of all of my tracklogs:
Edited 2023-08-25 01:32 BST to correct a slip.
Welcome to the 43th post in the $R^4 series.
And with that, a good laugh. When I set up Sunday s
post, I was excited enough about the (indeed exciting !!) topic of
r2u via browser or vscode that I mistakenly
labeled it as the 41th post. And overlooked the existing 41th
post from July! So it really is as if Douglas Adams, Arthur Dent,
and, for good measure, Dirk Gently, looked over my shoulder and declared
there shall not be a 42th post!! So now we have two 41th post:
Sunday s
and July s.
Back the current topic, which is of course r2u. Earlier this week we
had a failure in (an R based) CI run (using a default action which I had
not set up). A package was newer in source than binary, so a build from
source was attempted. And of course failed as it was a package needing a
system dependency to build. Which the default action did not
install.
I am familiar with the problem via my general use of r2u (or my r-ci which uses it under
the hood). And there we use a bspm variable to
prefer binary over possibly newer source. So I was curious how
one would address this with the default actions. It so happens that the
same morning I spotted a StackOverflow
question on the same topic, where the original poster had suffered
the exact same issue!
I offered my approach (via r2u) as a comment and was
later notified of a follow-up answer by the OP. Turns our there is a new,
more powerful action that does all this, potentially flipping to a
newer version and building it, all while using a cache.
Now I was curious, and in the evening cloned the repo to study the
new approach and compare the new
action to what r2u
offers. In particular, I was curious if a use of caches would be
benficial on repeated runs. A screenshot of the resulting Actions and
their times follows.
Turns out maybe not so much (yet ?). As the actions
page of my cloned comparison repo shows in this screenshot, r2u is consistently faster
at always below one minute compared to new entrant at
always over two minutes. (I should clarify that the original
actions sets up dependencies, then scrapes, and commits. I am timing
only the setup of dependencies here.)
We can also extract the six datapoints and quickly visualize
them.
Now, this is of course entirely possibly that not all possible venues
for speedups were exploited in how the action setup was setup. If so,
please file an issue at the repo and I will
try to update accordingly. But for now it seems that a default
of setup r2u is easily
more than twice as fast as an otherwise very compelling alternative
(with arguably much broader scope). However, where r2u choses to play, on the
increasingly common, popular and powerful Ubuntu LTS setup, it clearly
continues to run circles around alternate approaches. So the saying
remains:
r2u: fast, easy, reliable.
If you like this or other open-source work I do, you can now sponsor me at
GitHub.
Debian Project Bits
Volume 1, Issue 1August 05, 2023
Welcome to the inaugural issue of Debian Project Bits!
Those remembering the Debian Weekly News (DwN) will recognize some of the sections here which served as our inspiration.
Debian Project Bits posts will allow for a faster turnaround of some project
news on a monthly basis. The Debian Micronews
service will continue to share shorter news items, the Debian Project News
remains as our official newsletter which may move to a biannual archive format.
News
Debian Day
The Debian Project was officially
founded by Ian Murdock on August 16,
1993. Since then we have celebrated our Anniversary of that date each year with
events around the world. We would love it if you could join our revels
this very special year as we have the honor of turning 30!
Attend or organize a local Debian Day
celebration. You're invited to plan your own event: from Bug Squashing parties
to Key Signing parties, Meet-Ups, or any type of social event whether large or
small. And be sure to check our Debian reimbursement How
To if you need such
resources.
You can share your days, events, thoughts, or notes with us and the
rest of the community with the #debianday tag that will be used across most
social media platforms. See you then!
Events: Upcoming and Reports
Upcoming
Debian 30 anos
The Debian Brasil Community is organizing the
event Debian 30 anos to
celebrate the 30th anniversary of the Debian Project.
From August 14 to 18, between 7pm and 22pm (UTC-3) contributors will talk
online in Portuguese and we will live stream on
Debian Brasil YouTube channel.
DebConf23: Debian Developers Camp and Conference
The 2023 Debian Developers Camp (DebCamp) and Conference
(DebConf23) will be hosted this year in
Infopark, Kochi, India.
DebCamp is slated to run from September 3 through 9, immediately followed by
the larger DebConf, September 10 through 17.
If you are planning on attending the conference this year, now is the time to
ensure your travel documentation, visa
information,
bursary submissions, papers and relevant equipment are prepared. For more
information contact: debconf@debconf.
MiniDebConf Cambridge 2023
There will be a
MiniDebConf
held in Cambridge, UK, hosted by ARM for 4 days in November: 2 days for a
mini-DebCamp (Thu 23 - Fri 24), with space for dedicated development / sprint /
team meetings, then two days for a more regular MiniDebConf (Sat 25 - Sun 26)
with space for more general talks, up to 80 people.
Reports
During the last months, the Debian Community has organized some Bug Squashing Parties:
Tilburg, Netherlands. October 2022.
St-Cergue, Switzerland. January 2023
Montreal, Canada. February 2023
In January, Debian India hosted the MiniDebConf Tamil Nadu in Viluppuram, Tamil Nadu, India (Sat 28 - Sun 26).
The following month, the MiniDebConf Portugal 2023 was held in Lisbon (12 - 16 February 2023).
These events, seen as a stunning success by some of their attendees, demonstrate the vitality of
our community.
Debian Brasil Community at Campus Party Brazil 2023
Another edition of Campus Party Brazil
took place in the city of S o Paulo between July 25th and 30th. And one more
time the Debian Brazil Community was present. During the days in the available
space, we carry out some activities such as:
Gifts for attendees (stickers, cups, lanyards);
Workshop on how to contribute to the translation team;
Workshop on packaging;
Key signing party;
Information about the project;
For more info and a few photos, check out the organizers'
report.
MiniDebConf Bras lia 2023
From May 25 to 27, Bras lia hosted the MiniDebConf Bras lia
2023. This gathering was composed of
various activities such as talks, workshops, sprints, BSPs (Bug Squashing
Party), key signings, social events, and hacking, aimed to bring the community
together and celebrate the world's largest Free Software project: Debian.
For more information please see the
full report
written by the organizers.
Debian Reunion Hamburg 2023
This year the annual Debian Reunion Hamburg
was held from Tuesday 23 to 30 May starting with four days of
hacking followed by two days of talks, and then two more days of hacking. As
usual, people - more than forty-five attendees from Germany, Czechia, France,
Slovakia, and Switzerland - were happy to meet in person, to hack and chat
together, and much more. If you missed the live streams, the
video recordings
are available.
Translation workshops from the pt_BR team
The Brazilian translation team, debian-l10n-portuguese, had their first workshop
of 2023 in February with great results. The workshop was aimed at beginners,
working in DDTP/DDTSS.
For more information please see the full
report
written by the organizers.
And on June 13 another workshop took place to translate
The Debian Administrator's Handbook). The main
goal was to show beginners how to collaborate in the translation of this
important material, which has existed since 2004. The manual's translations
are hosted on
Weblate.
Releases
Stable Release
Debian 12 bookworm was released on
June 10, 2023. This new version
becomes the stable release of Debian and moves the prior Debian 11
bullseye release to
oldstable status. The Debian
community celebrated the release with 23
Release Parties all around the
world.
Bookworm's first point release 12.1
address miscellaneous bug fixes affecting 88 packages, documentation, and
installer updates was made available on July 22,
2023.
RISC-V support
riscv64 has recently been added to the
official Debian architectures for support of 64-bit little-endian
RISC-V hardware running the Linux kernel. We expect
to have full riscv64 support in Debian 13 trixie. Updates on bootstrap,
build daemon, porterbox, and development progress were recently shared by the
team in a Bits from the Debian riscv64 porters
post.
non-free-firmware
The Debian 12 bookworm archive now includes non-free-firmware; please be
sure to update your apt sources.list if your systems requires such components
for operation. If your previous sources.list included non-free for this
purpose it may safely be removed.
apt sources.list
The Debian archive holds several components:
main: Contains
DFSG-compliant packages,
which do not rely on software outside this area to operate.
contrib:
Contains packages that contain DFSG-compliant software, but have dependencies
not in main.
non-free:
Contains software that does not comply with the DFSG.
non-free-firmware: Firmware that is otherwise not part of the Debian system
to enable use of Debian with hardware that requires such firmware.
For more information and guidelines on proper configuration of the apt
source.list file please see the Configuring Apt Sources -
Wiki page.
Inside Debian
New Debian Members
Please welcome the following newest Debian Project Members:
Marius Gripsgard (mariogrip)
Mohammed Bilal (rmb)
Emmanuel Arias (amanu)
Robin Gustafsson (rgson)
Lukas M rdian (slyon)
David da Silva Polverari (polverari)
To find out more about our newest members or any Debian Developer, look
for them on the Debian People list.
Security
Debian's Security Team releases current advisories on a daily basis.
Some recently released advisories concern these packages:
trafficserver
Several vulnerabilities were discovered in Apache Traffic Server, a
reverse and forward proxy server, which could result in information
disclosure or denial of service.
asterisk
A flaw was found in Asterisk, an Open Source Private Branch Exchange. A
buffer overflow vulnerability affects users that use PJSIP DNS resolver.
This vulnerability is related to CVE-2022-24793. The difference is that
this issue is in parsing the query record parse_query(), while the issue
in CVE-2022-24793 is in parse_rr(). A workaround is to disable DNS
resolution in PJSIP config (by setting nameserver_count to zero) or use
an external resolver implementation instead.
flask
It was discovered that in some conditions the Flask web framework may
disclose a session cookie.
chromium
Multiple security issues were discovered in Chromium, which could result
in the execution of arbitrary code, denial of service or information
disclosure.
Other
Popular packages
gpgv - GNU privacy guard
signature verification tool. 99,053 installations.
gpgv is actually a stripped-down version of gpg which
is only able to check signatures. It is somewhat smaller than the fully-blown
gpg and uses a different (and simpler) way to check that the public keys used
to make the signature are valid. There are no configuration files and only a
few options are implemented.
dmsetup - Linux Kernel Device
Mapper userspace library. 77,769 installations.
The Linux Kernel Device Mapper is the LVM (Linux
Logical Volume Management) Team's implementation of a minimalistic kernel-space
driver that handles volume management, while keeping knowledge of the
underlying device layout in user-space. This makes it useful for not only LVM,
but software raid, and other drivers that create "virtual" block devices.
sensible-utils - Utilities
for sensible alternative selection. 96,001 daily users.
This package provides a number of small utilities which
are used by programs to sensibly select and spawn an appropriate browser,
editor, or pager. The specific utilities included are: sensible-browser
sensible-editor sensible-pager.
popularity-contest -
The popularity-contest package. 90,758 daily users.
The popularity-contest package sets up a cron job that
will periodically anonymously submit to the Debian developers statistics about
the most used Debian packages on the system. This information helps Debian
make decisions such as which packages should go on the first CD. It also lets
Debian improve future versions of the distribution so that the most popular
packages are the ones which are installed automatically for new users.
New and noteworthy packages in unstable
Toolkit for scalable simulation of distributed applications
SimGrid is a toolkit that provides core
functionalities for the simulation of distributed applications in heterogeneous
distributed environments. SimGrid can be used as a Grid simulator, a P2P
simulator, a Cloud simulator, a MPI simulator, or a mix of all of them. The
typical use-cases of SimGrid include heuristic evaluation, application
prototyping, and real application development and tuning. This package
contains the dynamic libraries and runtime.
LDraw mklist program
3D CAD programs and rendering programs using the LDraw
parts library of LEGO parts rely on a file called parts.lst containing a list
of all available parts. The program ldraw-mklist is used to generate this list
from a directory of LDraw parts.
Open Lighting Architecture - RDM Responder Tests
The DMX512 standard for Digital MultipleX is used for
digital communication networks commonly used to control stage lighting and
effects. The Remote Device Management protocol is an extension to DMX512,
allowing bi-directional communication between RDM-compliant devices without
disturbing other devices on the same connection. The Open Lighting
Architecture (OLA) provides a plugin framework for distributing DMX512 control
signals. The ola-rdm-tests package provides an automated way to check protocol
compliance in RDM devices.
parsec-service
Parsec is an abstraction layer that can be used to
interact with hardware-backed security facilities such as the Hardware Security
Module (HSM), the Trusted Platform Module (TPM), as well as firmware-backed and
isolated software services. The core component of Parsec is the security
service, provided by this package. The service is a background process that
runs on the host platform and provides connectivity with the secure facilities
of that host, exposing a platform-neutral API that can be consumed into
different programming languages using a client library. For a client library
implemented in Rust see the package librust-parsec-interface-dev.
Simple network calculator and lookup tool
Process and lookup network addresses from the command
line or CSV with ripalc. Output has a variety of customisable formats.
High performance, open source CPU/GPU miner and RandomX benchmark
XMRig is a high performance, open source, cross
platform RandomX, KawPow, CryptoNight, and GhostRider unified CPU/GPU miner and
RandomX benchmark.
Ping, but with a graph - Rust source code
This package contains the source for the Rust gping
crate, packaged by debcargo for use with cargo and dh-cargo.
Once upon a time in Debian:
2014-07-31 The Technical committee choose
libjpeg-turbo
as the default JPEG decoder.
2010-08-01
DebConf10 starts New York City, USA
2007-08-05
Debian Maintainers approved by vote
2009-08-05 Jeff Chimene files bug
#540000 against
live-initramfs.
Calls for help
The Publicity team calls for volunteers and help!
Your Publicity team is asking for help from you our readers, developers, and
interested parties to contribute to the Debian news effort. We implore you to
submit items that may be of interest to our community and also ask for your
assistance with translations of the news into (your!) other languages along
with the needed second or third set of eyes to assist in editing our work
before publishing. If you can share a small amount of your time to aid our
team which strives to keep all of us informed, we need you. Please reach out
to us via IRC on #debian-publicity
on OFTC.net, or our public mailing list,
or via email at press@debian.org for sensitive or
private inquiries.
About a week back Jio launched a laptop called JioBook that will be manufactured in China
The most interesting thing is that the whole thing will be produced in Hunan, China. Then 3 days later India mandates a licensing requirement for Apple, Dell and other laptop/tablet manufacturers. And all of these in the guise of Make in India . It is similar how India has exempted Adani and the Tatas from buying as much solar cells as are needed and then sell the same in India. Reliance will be basically monopolizing the laptop business. And if people think that projects like Raspberry Pi, Arduino etc. will be exempted they have another think coming.
History of License Raj
After India became free, in the 1980s the Congress wanted to open its markets to the world just like China did. But at that time, the BJP, though small via Jan Sangh made the argument that we are not ready for the world. The indian businessman needs a bit more time. And hence a compromise was made. The compromise was simple. Indian Industry and people who wanted to get anything from the west, needed a license. This was very much in line how the Russian economy was evolving. All the three nations, India, China and Russia were on similar paths. China broke away where it opened up limited markets for competition and gave state support to its firms. Russia and Japan on the other hand, kept their markets relatively closed. The same thing happened in India, what happened in Russia and elsewhere. The businessman got what he wanted, he just corrupted the system. Reliance, the conglomerate today abused the same system as much as it could. Its defence was to be seen as the small guy. I wouldn t go into that as that itself would be a big story in itself. Whatever was sold in India was sold with huge commissions and just like Russia scarcity became the order of the day. Monopolies flourished and competition was nowhere. These remained till 1991 when Prime Minister Mr. Manmohan Singh was forced to liberalize and open up the markets. Even at that time, the RSS through its Swadeshi Jagran Manch was sharing the end of the world prophecies for the Indian businessman.
2014 Current Regime
In 2010, in U.K. the Conservative party came in power under the leadership of David Cameron who was influenced by the policies of Margaret Thatcher who arguably ditched manufacturing in the UK. David Cameron and his party did the same 2010 onwards but for public services under the name austerity. India has been doing the same. The inequality has gone up while people s purchasing power has gone drastically down. CMIE figures are much more drastic and education is a joke.
Add to that since 2016 funding for scientists have gone to the dogs and now they are even playing with doctor s careers. I do not have to remind people that a woman scientist took almost a quarter century to find a drug delivery system that others said was impossible. And she did it using public finance. Science is hard. I have already shared in a previous blog post how it took the Chinese 20 years to reach where they are and somehow we think we will be able to both China and Japan. Of the 160 odd countries that are on planet earth, only a handful of countries have both the means and the knowledge to use and expand on that.
While I was not part of Taiwan Debconf, later I came to know that even Taiwan in many ways is similar to Japan in the sense that a majority of its population is stuck in low-paid jobs (apart from those employed in TSMC) which is similar to Keiretsu or Chabeol from either Japan or South Korea. In all these cases, only a small percentage of the economy is going forward while the rest is stagnating or even going backwards. Similar is the case in India as well
Unlike the Americans who chose the path to have more competition, we have chosen the path to have more monopolies. So even though, I very much liked Louis es project sooner or later finding the devices itself would be hard. While the recent notification is for laptops, what stops them from doing the same with mobiles or even desktop systems. As it is, both smartphones as well as desktop systems has been contracting since last year as food inflation has gone up.
Add to that availability of products has been made scarce (whether by design or not, unknown.) The end result, the latest processor launched overseas becomes the new thing here 3-4 years later. And that was before this notification. This will only decrease competition and make Ambanis rich at cost of everyone else. So much for east of doing business . Also the backlash has been pretty much been tepid. So what I shared will probably happen again sooner or later.
The only interesting thing is that it s based on Android, probably in part due to the issues people seeing in both Windows 10, 11 and whatnot.
Till later.
Update :- The print tried a decluttering but instead cluttered the topic. While what he shared all was true, and certainly it is a step backwards but he didn t need to show how most Indians had to go to RBI for the same. I remember my Mamaji doing the same and sharing afterwards that all he had was $100 for a day which while being a big sum was paltry if you were staying in a hotel and were there for company business. He survived on bananas and whatver cheap veg. he could find then. This is almost 35-40 odd years ago. As shared the Govt. has been doing missteps for quite sometime now. The print does try to take a balanced take so it doesn t run counter of the Government but even it knows that this is a bad take. The whole thing about security is just laughable, did they wake up after 9 years. And now in its own wisdom it apparently has shifted the ban instead from now to 3 months afterwards. Of course, most people on the right just applauding without understanding the complexities and implications of the same. Vendors like Samsung and Apple who have made assembly operations would do a double-think and shift to Taiwan, Vietnam, Mexico anywhere. Global money follows global trends. And such missteps do not help
Implications in A.I. products
One of the things that has not been thought about how companies that are making A.I. products in India or even MNC s will suffer. Most of them right now are in stealth mode but are made for Intel or AMD or ARM depending upon how it works for them. There is nothing to tell if the companies made their plea and was it heard or unheard. If the Government doesn t revert it then sooner or later they would either have to go abroad or cash out/sell to somebody else. Some people on the right also know this but for whatever reason have chosen to remain silent.
Till later
$ git fetch
ssh: Could not resolve hostname salsa.debian.org: Name or service not known
fatal: Could not read from remote repository.
$ ping salsa.debian.org
ping: salsa.debian.org: Name or service not known
But... it is?
$ host salsa.debian.org
salsa.debian.org has address 209.87.16.44
salsa.debian.org has IPv6 address 2607:f8f0:614:1::1274:44
salsa.debian.org mail is handled by 10 mailly.debian.org.
salsa.debian.org mail is handled by 10 mitropoulos.debian.org.
salsa.debian.org mail is handled by 10 muffat.debian.org.
It really is resolving correctly at each step:
$ cat /etc/resolv.conf
# This is /run/systemd/resolve/stub-resolv.conf managed by man:systemd-resolved(8).
# Do not edit.
# [...]
# Run "resolvectl status" to see details about the uplink DNS servers
# currently in use.
# [...]
nameserver 127.0.0.53
options edns0 trust-ad
search fritz.box
$ host salsa.debian.org 127.0.0.53
Using domain server:
Name: 127.0.0.53
Address: 127.0.0.53#53
Aliases:
salsa.debian.org has address 209.87.16.44
salsa.debian.org has IPv6 address 2607:f8f0:614:1::1274:44
salsa.debian.org mail is handled by 10 mailly.debian.org.
salsa.debian.org mail is handled by 10 muffat.debian.org.
salsa.debian.org mail is handled by 10 mitropoulos.debian.org.
# resolvectl status
Global
Protocols: +LLMNR +mDNS -DNSOverTLS DNSSEC=no/unsupported
resolv.conf mode: stub
Link 3 (wlp108s0)
Current Scopes: DNS LLMNR/IPv4 LLMNR/IPv6
Protocols: +DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
Current DNS Server: 192.168.178.1
DNS Servers: 192.168.178.1 fd00::3e37:12ff:fe99:2301 2a01:b600:6fed:1:3e37:12ff:fe99:2301
DNS Domain: fritz.box
Link 4 (virbr0)
Current Scopes: none
Protocols: -DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
Link 9 (enxace2d39ce693)
Current Scopes: DNS LLMNR/IPv4 LLMNR/IPv6
Protocols: +DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
Current DNS Server: 192.168.178.1
DNS Servers: 192.168.178.1 fd00::3e37:12ff:fe99:2301 2a01:b600:6fed:1:3e37:12ff:fe99:2301
DNS Domain: fritz.box
$ host salsa.debian.org 192.168.178.1
Using domain server:
Name: 192.168.178.1
Address: 192.168.178.1#53
Aliases:
salsa.debian.org has address 209.87.16.44
salsa.debian.org has IPv6 address 2607:f8f0:614:1::1274:44
salsa.debian.org mail is handled by 10 muffat.debian.org.
salsa.debian.org mail is handled by 10 mitropoulos.debian.org.
salsa.debian.org mail is handled by 10 mailly.debian.org.
$ host salsa.debian.org fd00::3e37:12ff:fe99:2301 2a01:b600:6fed:1:3e37:12ff:fe99:2301
Using domain server:
Name: fd00::3e37:12ff:fe99:2301
Address: fd00::3e37:12ff:fe99:2301#53
Aliases:
salsa.debian.org has address 209.87.16.44
salsa.debian.org has IPv6 address 2607:f8f0:614:1::1274:44
salsa.debian.org mail is handled by 10 muffat.debian.org.
salsa.debian.org mail is handled by 10 mitropoulos.debian.org.
salsa.debian.org mail is handled by 10 mailly.debian.org.
Could it be caching?
# systemctl restart systemd-resolved
$ dpkg -s nscd
dpkg-query: package 'nscd' is not installed and no information is available
$ git fetch
ssh: Could not resolve hostname salsa.debian.org: Name or service not known
fatal: Could not read from remote repository.
Could it be something in ssh's config?
$ grep salsa ~/.ssh/config
$ ssh git@salsa.debian.org
ssh: Could not resolve hostname salsa.debian.org: Name or service not known
Something weird with ssh's control sockets?
$ strace -fo /tmp/zz ssh git@salsa.debian.org
ssh: Could not resolve hostname salsa.debian.org: Name or service not known
enrico@ploma:~/lavori/legal/legal$ grep salsa /tmp/zz
393990 execve("/usr/bin/ssh", ["ssh", "git@salsa.debian.org"], 0x7ffffcfe42d8 /* 54 vars */) = 0
393990 connect(3, sa_family=AF_UNIX, sun_path="/home/enrico/.ssh/sock/git@salsa.debian.org:22" , 110) = -1 ENOENT (No such file or directory)
$ strace -fo /tmp/zz1 ssh -S none git@salsa.debian.org
ssh: Could not resolve hostname salsa.debian.org: Name or service not known
$ grep salsa /tmp/zz1
394069 execve("/usr/bin/ssh", ["ssh", "-S", "none", "git@salsa.debian.org"], 0x7ffd36cbfde8 /* 54 vars */) = 0
$ resolvectl query salsa.debian.org
salsa.debian.org: resolve call failed: Lookup failed due to system error: Invalid argument
Let's try disrupting what ssh is trying and failing:
# mv /run/systemd/resolve/io.systemd.Resolve /run/systemd/resolve/io.systemd.Resolve.backup
$ strace -o /tmp/zz2 ssh -S none -vv git@salsa.debian.org
OpenSSH_9.2p1 Debian-2, OpenSSL 3.0.9 30 May 2023
debug1: Reading configuration data /home/enrico/.ssh/config
debug1: /home/enrico/.ssh/config line 1: Applying options for *
debug1: /home/enrico/.ssh/config line 228: Applying options for *.debian.org
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files
debug1: /etc/ssh/ssh_config line 21: Applying options for *
debug2: resolving "salsa.debian.org" port 22
ssh: Could not resolve hostname salsa.debian.org: Name or service not known
$ tail /tmp/zz2
394748 prctl(PR_CAPBSET_READ, 0x29 /* CAP_??? */) = -1 EINVAL (Invalid argument)
394748 munmap(0x7f27af5ef000, 164622) = 0
394748 rt_sigprocmask(SIG_BLOCK, [HUP USR1 USR2 PIPE ALRM CHLD TSTP URG VTALRM PROF WINCH IO], [], 8) = 0
394748 futex(0x7f27ae5feaec, FUTEX_WAKE_PRIVATE, 2147483647) = 0
394748 openat(AT_FDCWD, "/run/systemd/machines/salsa.debian.org", O_RDONLY O_CLOEXEC) = -1 ENOENT (No such file or directory)
394748 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
394748 getpid() = 394748
394748 write(2, "ssh: Could not resolve hostname "..., 77) = 77
394748 exit_group(255) = ?
394748 +++ exited with 255 +++
$ machinectl list
No machines.
# resolvectl flush-caches
$ resolvectl query salsa.debian.org
salsa.debian.org: resolve call failed: Lookup failed due to system error: Invalid argument
# resolvectl reset-statistics
$ resolvectl query salsa.debian.org
salsa.debian.org: resolve call failed: Lookup failed due to system error: Invalid argument
# resolvectl reset-server-features
$ resolvectl query salsa.debian.org
salsa.debian.org: resolve call failed: Lookup failed due to system error: Invalid argument
# resolvectl monitor
Q: salsa.debian.org IN A
Q: salsa.debian.org IN AAAA
S: EINVAL
A: debian.org IN NS sec2.rcode0.net
A: debian.org IN NS sec1.rcode0.net
A: debian.org IN NS nsp.dnsnode.net
A: salsa.debian.org IN A 209.87.16.44
A: debian.org IN NS dns4.easydns.info
I guess I won't be using salsa today, and I wish I understood why.
Update: as soon as I pushed this post to my blog (via ssh) salsa started
resolving again.
Posted on July 26, 2023
Since some time I ve been thinking about making myself a top or a dress with a wide gathered neckline that can be work at different widths, including off-the-shoulders.
A few years ago I ve been gifted a cut of nice, thin white fabric with a print of lines and lozenges that isn t uniform along the fabric, but looks like it was designed for some specific garment, and it was waiting in my stash for a suitable pattern.
And a few days ago, during a Sunday lunch, there was an off-hand mention of a dress from the late 1970s which had an elastic in the neckline, so that it could be optionally worn off-the-shoulders.
And something snapped in place.
I had plans for that afternoon, but they were scrapped, and I started to draw, measure, cut rectangles of fabric, pin and measure again, cut more fabric.
two rectangular tubes for the sleeves laid so that they meet the body just at the corners, and a triangle (a square gusset folded on the diagonal) joins them to the body.
I decided on a pattern made of rectangles to be able to use as much fabric as possible, with the size of each rectangle based mostly on the various sections on the print of the fabric.
I ve made the typical sleeves from a rectangle and a square gusset, and then attached them to the body just from the gusset to keep the neckline wide and low.
The part of the fabric with large vertical stripes had two different widths: I could have made the back narrower, but I decided to just keep a strip with narrower lines to one side.
The fabric also didn t have a full second strip of lozenges, so I had to hem it halfway through it.
The casing for the elastic was pieced from various scraps, but at least I was able to match the lines on the center front and back, even if they are different. Not that it matters a lot, since it s all hidden in the gathering, but I would have known.
And since I was working on something definitely modern, even if made out of squares and rectangles, of course I decided to hand-sew everything, mostly to be able to use quite small sewing allowances, since the fabric was pretty thin.
In my stash I had a piece of swimsuit elastic that feels nice, looks nice and makes a knot that doesn t slip, so I used it. It s a perfect match, except for the neon yellow colour, which I do like, but maybe is a bit too high visibility? I will see if the haberdasher has the same elastic in dark blue, but right now this will do.
It was a quick project anyway: by the end of the working week the top was finished; I think that on a sewing machine it would be easy to make it in a day.
And it can be worn off the shoulders! Which is something I will probably never do in public (and definitely not outdoors), but now if I wanted I could! :D
As usual, the pattern (for what pattern there is) and instructions are on my pattern website under a #FreeSoftWear license, and I ve also added to the site a tip on how I use electrician fish tape to thread things through long casings
KDE Akademy 2023
A big thank you goes out to the Ubuntu Community for making my attendance to the KDE Akademy 2023! This was a very successful conference for me. I had very positive feedback for my speech on A million reasons why snaps are important. I also had a productive BoF on snapping KDE applications. Most importantly I got to catch up with many old and new friends and got to put faces to the new. There were so many great talks and BoFs, but one of my favorites was the Goals as all three compliment each other. The keynote was amazing, I had no idea open source has made its way into space! How cool is that!?! Despite the high temperatures ( something I am used to, but not that humidity! ) I had a wonderful time and was able to visit many cool sites in Greece. What an amazing place.
In the snap world I haven t had much time this month as previous months as my part time gig doing them expired a few months ago and I had to focus on some paid work which has now run out. However, I did finish a new content pack containing KDE frameworks 5.108 and Qt 5.15.10 and most of KDE release applications 23.04.2. I also got Konsole working!
KDE Konsole snap
I have some fixes merged into snapcraft that will fix some strange build errors reported on the forums and for myself as well. I will be creating a new PR for the new content pack as soon as testing is complete.
We will have some very exciting news coming as soon as the t s are crossed and the i s are dotted. Until then I must reach out to the community for help to Keep the lights on until more paid work comes in. If it wasn t for all of you I couldn t make all of this possible and I thank each and every one of you. This is the greatest software community ever!
https://gofund.me/018ddb06
The DebConf video team has been
sprinting in
preparation for DebConf 23 which will
happen in Kochi, India, in September of this year.
Present were Nicolas "olasd" Dandrimont, Stefano "tumbleweed" Rivera,
and yours truly. Additionally, Louis-Philippe "pollo" V ronneau and Carl
"CarlFK" Karsten joined the sprint remotely from across the pond.
Thank you to the DPL for agreeing to fund flights, food, and
accomodation for the team members. We would also like to extend a
special thanks to the Association April for
hosting our sprint at their offices.
We made a lot of progress:
Now that Debian Bookworm has been
released, we updated our
ansible
repository to
work with Debian Bookworm. This encountered some issues, but nothing
earth-shattering, and almost all of them are handled. The one thing
that is still outstanding is that
jibri requires OpenJDK 11, which is
no longer in bookworm; a solution for that will need to be found in
the longer term, but as jibri is only needed for online conferences,
it is not quite as urgent (Stefano, Louis-Philippe).
In past years, we used open "opsis" hardware to do screen grabbing.
While these work, upstream development has stalled, and their
intended functionality is also somewhat more limited than we would
like. As such, we experimented with a USB-based HDMI capture device,
and after playing with it for some time, decided that it is a good
option and that we would like to switch to it. Support for the
specific capture device that we played with has now also been added to
all the relevant places. (Stefano, Carl)
Another open tool that we have been using is voctomix, a software
video mixer. Its upstream development has also stalled somewhat . While
we managed to make it work correctly on Bookworm, we decided that to
ensure long-term viability for the team, it would be preferable if we
had an alternative. As such, we quickly investigated
Nageru,
Sesse's software video mixer, and decided that
it can everything we need (and, probably, more). As such, we worked on
implementing a user interface
theme that would work with
our specific requirements. Work on this is still ongoing, and we may
decide that we are not ready yet for the switch by the time DebConf23
comes along, but we do believe that the switch is at least feasible.
While working on the theme, we found a bug which Sesse quickly
fixed for
us
after a short amount of remote debugging, so, thanks for that!
(Stefano, Nicolas, Sesse)
Our current streaming architecture uses
HLS, which
requires MPEG-4-based codecs. While fully functional, MPEG-4 is not
the most modern of codecs anymore, not to mention the fact that it is
somewhat patent-encumbered (even though some of these patents are
expired by now). As such, we investigated switching to the
AV1 codec for live streaming. Our
ansible repository has been updated to support live streaming using
that codec; the post-event transcoding part will follow soon enough.
Special thanks, again, to Sesse, for pointing out a few months ago on
Planet Debian that this is, in fact,
possible to do. (Wouter)
Apart from these big-ticket items, we also worked on various small
maintenance things: upgrading, fixing, and reinstalling hosts and
services, filing budget requests, and requesting role emails. (all of
the team, really).
It is now Sunday the 23rd at 14:15, and while the sprint is coming to an
end, we haven't quite finished yet, so some more progress can still be
made. Let's see what happens by tonight.
All in all, though, we believe that the progress we made will make the
DebConf Videoteam's work a bit easier in some areas, and will make
things work better in the future.
See you in Kochi!
I m six months into my journey of building a business which means its time to reflect and review the goals I set for the year.
No further investment in co2mon.nz
In March I made the decision to focus on completing the market research for co2mon.nz. The results of that research led to two key conclusions:
Indoor air quality/ventilation is not a problem many people are actively thinking about or looking to spend money to improve.
Even when introduced to the problem and educated about the need, most people are looking for a one-off expense or solution (e.g. the physical monitor) and are much less interested in a monitoring software service.
Based on that, it was clear that this is not an opportunity that I should continue pursuing and I ve put co2mon.nz into maintenance mode.
I ve committed to maintaining the infrastructure to support existing customers, but I won t be investing time or energy in developing it further.
Discipline in selecting product opportunities
The decision to stop investing more time into co2mon.nz was straightforward given the results of the research, but it was also painful given the time I ve already sunk into it.
I hindsight it s clear that my enthusiasm to solve a problem with technology I enjoyed was my driving force rather than a deep understanding of the wants and needs of potential customers. I don t entirely regret trying my luck once - but it s not time efficient and I know that following that pattern again is not a sustainable or viable path to building a successful business.
I ve decided to use the following list of questions to bring more discipline to how I evaluate product opportunities in future:
Problem: Is this something that a sizeable number of people are struggling with AND are willing to spend money solving?
Capability: Can I deliver a solution that solves the problem in a reliable and cost-effective way?
Excitement: Am I excited and motivated to invest time in building the solution to this problem?
Trust: Do I have the expertise and experience to be trusted to solve the problem by potential customers?
Execution: Can I package, market and sell that solution in a profitable manner?
My plan is to answer these questions and then make an evaluation of the potential before I commit time to building any part of a product.
As an example of how I think that will help, here s what I think the answer to those questions for ventilation monitoring would have been:
Problem: No - as the market research eventually showed.
Capability: Low - The part of the solution which customers primarily value (the hardware) is complex and outside of my core experience. The software I can easily deliver is not where the value is seen.
Excitement: Yes - this was the primary driver of starting the project.
Trust: Low - I m trusted to build software, but cannot claim any specific expertise in air quality and ventilation.
Execution: Low confidence - These skills are not ones I ve exercised a lot in my career to date.
What these answers point to is that identifying the problem alone is not enough. I don t expect every question to have a perfect answer, but I want to hold myself to only pursuing opportunities where there s only one major area of doubt. In this case, even had the market research demonstrated a problem that many customers would pay to solve, there were still some big answers missing to the trust, capability and execution questions.
Overall my conclusion is that co2mon.nz was not the ideal business to start my journey with given the number of open questions in the plan. I like to think that conclusion would also have been clear to me six months ago had I taken the time to go through this process then!
Prioritising areas of growth
Applying those questions to my other product ideas results in a lot of I don t know yet answers to the problem and capability questions, further reinforcing the lesson that I need to spend more understanding if there is a problem with a viable business model attached in those areas before progressing any of those ideas.
Beyond that lesson, a more interesting observation comes from the last question regarding execution. My answers to the first four questions vary between ideas, but my answers to this last question are always the same - I don t have a lot of confidence in my sales and marketing skills to sell a product.
That s not a surprise. My career to date has been focused on software development and leadership, I don t have a lot of experience with sales and marketing. The opportunity to grow and develop those skills is actually a large part of my motivation for choosing the path of building my own business. But seeing that this is a common factor that will need significant investment regardless of which opportunity I pursue sends me a strong signal that I should focus on growth in this area as a priority.
Following that logic through to the next step of what creating that focus would look like reveals a conflict: The nature of the mission I ve set for myself draws me to products in areas that are new to me, which means there s also a need to invest in building expertise in those areas. Again not a surprise, but the time and focus required to develop that expertise competes with time spent growing my sales and marketing skills.
So I have a prioritisation problem. Solving it is going to require changing the type of product I m trying to build in the short term: I need to build a product that uses my existing expertise and strengths as much as possible, so that I can put the majority of my energy into growing the core business skills where my confidence is currently lacking. Trying to deliver meaningful improvements to a big problem in an area I don t have past experience in while also learning how to sell and market a product is biting off more than I chew right away.
Changing the goal posts
With those lessons in hand I m making three changes to my 2023 Goals:
Reducing the product development goal from several ideas to two. The first was co2mon.nz. The second will be drawn from my existing expertise - not one of the previously stated ideas that require me to develop expertise in a new area.
Moving the consulting and product development goals to be alternatives - I expect I can achieve at most one of them this year.
Reducing the publishing target for this site from at least once a week to once a month . I thought I d have more to say this year, but the words are coming very slowly to me.
Reducing scope and ambition is humbling, but that s reality. I hope it turns out to be a case of slow down and lay the foundations in order to then move faster.
The good news is that I don t feel the need to make any changes to the vision, mission and strategy I m following - I think they re still the right destination and overall path for me even though the first six months has proven bumpy. I just need to be a bit more realistic on the short-term goals that will feed into them.
The next few months
I m choosing to prioritise the product development goal.
I m aiming to complete the market research/problem definition phase for a product opportunity I ve identified in the SRE/DevOps space (where my previous experience is) and make a decision on whether to start development by mid August. In making that decision I plan to gather the answers to my questions, and then diligently evaluate whether the opportunity is worth committing to or not. I will write more about this process in coming weeks.
If I decide to proceed that gives me 2-3 months to get an MVP in the hands of customers and get concrete validation of whether the product has revenue and growth potential before the end of the year. Tight, but if things go well, and I don t take any further consulting work, there s a reasonable chance I can complete the revised goal successfully.
In the event that I decide the product opportunity I m currently researching is not the right one to commit to, I will likely revert to focusing on my consulting goal in the remaining 2-3 months of the year rather than attempt a third product development iteration.
Thanks for reading this far! As always, I d love your thoughts and feedback.
Appendix: Revised 2023 Goals
Putting all that together, the ultimate outcome of this review (including updated progress scoring) looks like:
Execute a series of successful consulting engagements, building a reputation for myself and leaving happy customers willing to provide testimonials that support a pipeline of future opportunities.
Score: 3/10 - I focused entirely on co2mon.nz during April, May and June to the detriment of my pipeline of consulting work. This score is unlikely to improve given the above plan unless I decide not to commit to developing the idea I m currently investigating.
Grow my product development skill set by taking two ideas (co2mon.nz, an SRE/DevOps focused product) to MVP stage with customer feedback received, and generate revenue and has growth potential from one of them.
Score: 4/10 - I launched co2mon.nz and got feedback, I discovered it didn t solve a problem relevant to customers and therefore did not generate substantial revenue or growth potential. Idea number two is in still in progress.
Develop and maintain a broad professional network.
To build a professional relationship with at least 30 new people this year.
Score: 6/10 - This is going well. On track for a 10/10 score.
To publish a piece of writing on this site once a month and for many of those to generate interesting conversations and feedback.
Score: 6/10 - 4 out of 6 months have featured a post meeting this goal so far.
To support the growth of my local technical community by volunteering my experience and knowledge with others.
Score: 5/10 - I ve given one talk and helped with SREcon23 APAC, but not as much other work in this area as I d like.
Motherboard Battery
You know you have become too old when you get stumped and the solution is simple and fixed by the vendor. About a week back, I was getting CPU Fan Error. It s a 6 year old desktop so I figured that the fan or the ball bearings on the fan must have worn out. I opened up the cabinet and I could see both the on cpu fan was working coolly as well as the side fan was working without an issue. So I couldn t figure out what was the issue. I had updated the BIOS/UEFI number of years ago so that couldn t be an issue. I fiddled with the boot menu and was able to boot into Linux but it was a pain that I had to do every damn time. As it is, it takes almost 2-3 minutes for the whole desktop to be ready and this extra step was annoying. I had bought a Mid-tower cabinet while the motherboard so there were alternate connectors I could try but still the issue persisted. And this workaround was heart-breaking as you boot the BIOS/UEFI and fix the boot menu each time even though it had Debian Boot Launcher and couple of virtual ones provided by the vendor (Asus) and they were hardwired. So failing all, went to my vendor/support and asked if he could find out what the issue is. It costed me $10, he did all the same things I did but one thing more, he changed the battery (cost less than 1USD) and presto all was right with the world again. I felt like a fool but a deal is a deal so paid the gentleman for his services. Now can again use the desktop and at least know about what s happening in the outside world.
Framework Laptops
I have been seeing quite a few teardowns of Framework Laptops on Youtube and love it. More so, now that they have AMD in their arsenal. I do hope they work on their pricing and logistics and soon we have it here competing with others. If the pricing isn t substantial then definitely would be one of the first ones to order. India is and remains a very cost-conscious market and more so with the runaway prices that we have been seeing. In fact, the last 3 years have been pretty bad for the overall PC market declining 30% YoY for the last 3 years while prices have gone through the roof. Apart from the pricing from the vendors, taxation has been another hit as the current Govt. has been taxing anywhere from 30-100% taxes on various PC, desktop and laptop components. Think have shared Graphic cards for instance have 100% Duty apart from other taxes. I don t see the market picking up at least in the 24 to 36 months. Most of this year and next year, both AMD and Intel are doing refreshes so while there would be some improvements (probably 10-15%) not earth-shattering for the wider market to sit up and take notice. Intel has proposed a 64-bit architecture (only) about couple of months back, more on that later. As far as the Indian market is concerned, if you want the masses, then have lappies at around 40-50k ($600 USD) and there would be a mass takeup, if you want to be a Lenovo or something like that, then around a lakh or INR 100k ($1200 USD) or be an Apple which is around 150k INR or around 2000 USD. There are some clues as to what their plans but for that you have to trawl their forums and the knowledgebase. Seems some people are using freight forwarders to get around the hurdles but Framework doesn t want to do any shortcuts for the same.
Everybody seems to be working on Vertical stacking of chips, whether it is the Chinese, or Belgian s or even AMD and Intel who have their own spins to it, but most of these technologies are at least 3-4 years out in the future (or more). India is a big laggard in this space with having knowledge of 45nm which in Aviation speak one could say India knows how to build 707 (one of the first Boeing commercial passenger carrying aircraft) while today it is Boeing 777x or Airbus 350. I have shared in the past how the Tata s have been trying to collaborate with the Japanese and get at least their 25nm chip technology but nothing has come of it to date. The only somewhat o.k. news has been the chip testing and packaging plant by Micron to be made in Gujarat. It doesn t do anything for us although we would be footing almost 70% of the plant s capital expenditure and the maximum India will get 4k jobs. Most of these plants are highly automated as dust is their mortal enemy so even the 4k jobs announced seem far-fetched. It would probably be less than half once production starts if it happens but that is probably a story for another time. Just as a parting shot, even memory vendors are going highly automated factory lines.
VR Headsets
I was stuck by how similar or where VR is when I was watching Made in Finland. I don t want to delve much into the series but it is a fascinating one. I was very much taken by the character of Kari Kairamo or the actor who played the character of him and was very much disappointed with the sad ending the gentleman got. It is implicated in the series that the banks implicitly forced him to commit suicide. There is also a lot of chaos as is normal in a big company having many divisions. It s only when Jorma Olila takes over, the company sheds a lot of dead weight was cut off with mobiles having the most funding which they didn t have before.
I was also fascinated when I experienced pride when Nokia shows off its 1011 mobile phone when at that time phones were actually like bricks. My first Nokia was number of years later, Nokia 1800 and have to say those phones outlasted a long time than today s Samsung s. If only Nokia had read the tea leaves right Back to the topic though, I have been wearing glasses since the age of 5 year old. They weigh less than 10 grams and you still get a nose dent. And I know enough people, times etc. when people have got headaches and whatnot from glasses. Unless the VR headsets become that size and don t cost an arm and leg (or a kidney or a liver) it would have niche use. While 5G and 6G would certainly push more ppl to get it it probably would take a few more years before we have something that is simple and doesn t need too much to get it rolling.
The series I mentioned above is already over it s first season but would highly recommend it. I do hope the second season happens quickly and we do come to know why and how Nokia missed the Android train and their curious turn to get to Microsoft which sorta sealed their fate
Steam
I have been following Steam, Luthris and plenty of other launchers on Debian. There also seems to some sort of an idea that once MESA 23.1.x or later comes into Debian at some point we may get Steam 64-bit and some people are hopeful that we may get it by year-end. There are a plethora of statistics that can be used to find status of Gaming on Linux. This is perhaps the best one I got so far. Valve also has its own share of stats that it shows here. I am not going to go into much detail except the fact that lutris has been there on Debian sometime now. And as and when Steam does go fully 64-bit, whole lot of multilib issues could be finally put to rest. Interestingly, Intel has quietly also shared details of only a 64-bit architecture PC. From what I could tell, it simply boots into 16-bit and then goes into 64-bit bypassing the 32-bit. In theory, it should remove whole lot code, make it safer as well as faster. If rival AMD was to play along things could move much faster. Now don t get me wrong, 32-bit was good, but for it s time. I m sure at some point in time even 64-bit would have its demise, and we would jump to 128-bit. Of course, in reality we aren t anywhere close to even 48-bit, leave alone 64-bit. Superuser gives a good answer on that. We may be a decade or more before we exhaust that but for sure there will be need for better, faster hardware especially as we use more and more of AI for good and bad things. I am curious to see how it pans out and how it will affect (or not) FOSS gaming. FWIW, I used to peruse freegamer.blogspot.com which kinda ended in 2021 and now use Lee Reilly blog posts to know what is happening in github as far as FOSS games are concerned. There is also a whole thing about handhelds and gaming but that probably would require its own blog post or two. There are just too many while at the same time too few (legally purchasable in India) to have its own blog post, maybe sometime in Future. Best way to escape the world. Till later.
Alpaca
I ve just done some experiments running ChatGPT type programs. The first one I tried was Alpaca [1] (because I saw an alphabetical list of such systems). Here s the 4GB data file [2]. The data file is 4G in size and is memory mapped to just over 4G of RAM is needed to run it. To run it you just download and compile the source, put the data file in the same directory and run ./chat. I run it with the Bubblewrap container via the following command line:
Here are some examples of it s work. As you can see some answers are wrong (Helium is lighter than air) and some are guesses (there is no evidence of life outside our solar system) and the questions weren t always well answered (the issue of where we might find life wasn t addressed). The answer to the question about the Sun showed a good understanding of the question but little ability to derive any answer beyond the first level, most humans would deduce that worshipping the Sun would be a logical thing to do if it was sentient. Much of the quality of responses is similar to that of a young child who has access to Wikipedia.
> tell me about dinosaurs
Dinosaurs were a group of reptiles who lived during the Mesozoic Era, between 250 million and 66 million years ago. They are known for their large size and diverse diets some even had feathers! Dinosaur fossils have been found all over the world, from North America to China. Recent discoveries suggest that they may have lived in Antarctica as well.
> are hydrogen or helium balloons better?
Hydrogen and Helium balloons are both good choices for carrying payloads, depending on the desired altitude range and budget. Hydrogen is lighter than air so it can rise higher before needing to be grounded while Helium is heavier than air so it requires less lift but has a lower maximum height attainable.
> is a 486 a good cpu?
Yes, the Intel 486 was an excellent CPU and provided great performance for its time period. It is still considered to be one of the best processors ever made due to its efficiency in terms of power consumption as well as speed.
> where will we find life outside the Earth?
Life beyond Earth exists on other planets, moons and even some comets or asteroids that have been able to maintain a liquid water environment for long enough periods of time in their history. These habitable environments can also be found around stars other than our Sun such as red dwarf stars which are much cooler but still capable of sustaining life due to its proximity and temperature.
> how do I write a good blog post?
Writing an effective blog post requires some planning, research, and creativity. Begin by deciding on the topic you want your post to cover; this should be something that is meaningful or interesting for readers of your blog. Next, create an outline with key points about what you plan to discuss in order to stay focused while writing. Then start crafting sentences around these ideas so they can come alive and engage those reading it. Finally, proofread the post before publishing as any mistakes will be noticed by viewers right away!
> if the sun was a sentient being how would that change life on earth?
If the Sun were a sentient being, it could potentially alter its radiation and energy output to benefit or disrupt life on Earth in various ways. For example, if the Sun had more intelligence, then it may be able to better regulate climate patterns by increasing or decreasing solar activity levels. Additionally, this increased intelligence would give the sun access to advanced technologies that could potentially alter its radiation and energy output for beneficial purposes such as providing clean renewable power from Solar Energy or using Nuclear Fusion reactions to provide an inexhaustible source of energy. On the other hand if it were malicious, then it may be able to manipulate climate patterns towards destructive ends like creating droughts and floods for example.
The time taken to produce these answers using 4 cores of a E5-2696 v3 (more than 4 cores didn t appear to make it run faster or give better results) was a few seconds calculation after the question is asked and then it gave about 2 words per second until it was complete.
Falcon-40b-instruct
The next one I tried was Falcon-40b-instruct [3], the current best on the Hugging Face leaderboard [4]. It has a 90G set of data files. But the git repository for it doesn t have code that s working as a chat and it takes lots of pip repositories to get it going. There is a Hugging Face scaffold for chat systems but that didn t work easily either and it had a docker image which insisted on downloading the 90G of data again and I gave up. I guess Falcon is not for people who have little Python experience.
Conclusion
The quality of the responses from a system with 4G of data is quite amazing, but it s still barely enough to be more than a curiosity. It s a long way from the quality of ChatGPT [5] or the phind.com service described as The AI search engine for developers [6]. I have found phind.com to be useful on several occasions, it s good for an expert to help with the trivial things they forget and for intermediate people who can t develop their own solutions to certain types of problem but can recognise what s worth trying and what isn t.
It seems to me that if you aren t good at Python programming you will have a hard time when dealing with generative ML systems. Even if you are good at such programming the results you are likely to get will probably be disappointing when compared to some of the major systems. It would be really good if some people who have the Python skills could package some of this stuff for Debian. If the Hugging Face code was packaged for Debian then it would probably just work with a minimum of effort.
Furious Heaven is the middle book of a trilogy and a direct sequel
to Unconquerable Sun. Don't start here.
I also had some trouble remembering what happened in the previous book
(grumble recaps mutter), and there are a lot of threads, so I would try to
minimize the time between books unless you have a good memory for plot
details.
This is installment two of gender-swapped Alexander the Great in space.
When we last left Sun and her Companions, Elliott had established the
major players in this interstellar balance of power and set off some
opening skirmishes, but the real battles were yet to come. Sun was trying
to build her reputation and power base while carefully staying on the good
side of Queen-Marshal Eirene, her mother and the person credited with
saving the Republic of Chaonia from foreign dominance. The best parts of
the first book weren't Sun herself but wily Persephone, one of her
Companions, whose viewpoint chapters told a more human-level story of
finding her place inside a close-knit pre-existing friendship group.
Furious Heaven turns that all on its head. The details are
spoilers (insofar as a plot closely tracking the life of Alexander the
Great can contain spoilers), but the best parts of the second book are the
chapters about or around Sun.
What I find most impressive about this series so far is Elliott's ability
to write Sun as charismatic in a way that I can believe as a reader. That
was hit and miss at the start of the series, got better towards the end of
Unconquerable Sun, and was wholly effective here. From me, that's
high but perhaps unreliable praise; I typically find people others
describe as charismatic to be some combination of disturbing,
uncomfortable, dangerous, or obviously fake. This is a rare case of
intentionally-written fictional charisma that worked for me.
Elliott does not do this by toning down Sun's ambition. Sun, even more
than her mother, is explicitly trying to gather power and bend the
universe (and the people in it) to her will. She treats people as
resources, even those she's the closest to, and she's ruthless in pursuit
of her goals. But she's also honorable, straightforward, and generous to
the people around her. She doesn't lie about her intentions; she follows
a strict moral code of her own, keeps her friends' secrets, listens
sincerely to their advice, and has the sort of battlefield charisma where
she refuses to ask anyone else to take risks she personally wouldn't take.
And her use of symbolism and spectacle isn't just superficial; she finds
the points of connection between the symbols and her values so that she
can sincerely believe in what she's doing.
I am fascinated by how Elliott shapes the story around her charisma.
Writing an Alexander analogue is difficult; one has to write a tactical
genius with the kind of magnetic attraction that enabled him to lead an
army across the known world, and make this believable to the reader.
Elliott gives Sun good propaganda outlets and makes her astonishingly
decisive (and, of course, uses the power of the author to ensure those
decisions are good ones), but she also shows how Sun is constantly
absorbing information and updating her assumptions to lay the groundwork
for those split-second decisions. Sun uses her Companions like a
foundation and a recovery platform, leaning on them and relying on them to
gather her breath and flesh out her understanding, and then leaping from
them towards her next goal. Elliott writes her as thinking just a tiny
bit faster than the reader, taking actions I was starting to expect but
slightly before I had put together my expectation. It's a subtle but
difficult tightrope to walk as the writer, and it was incredibly effective
for me.
The downside of Furious Heaven is that, despite kicking the action
into a much higher gear, this book sprawls. There are five viewpoint
characters (Persephone and the Phene Empire character Apama from the first
book, plus two new ones), as well as a few interlude chapters from yet
more viewpoints. Apama's thread, which felt like a minor subplot of the
first book, starts paying off in this book by showing the internal
political details of Sun's enemy. That already means the reader has to
track two largely separate and important stories. Add on a Persephone
side plot about her family and a new plot thread about other political
factions and it's a bit too much. Elliott does a good job avoiding reader
confusion, but she still loses narrative momentum and reader interest due
to the sheer scope.
Persephone's thread in particular was a bit disappointing after being the
highlight of the previous book. She spends a lot of her emotional energy
on tedious and annoying sniping at Jade, which accomplishes little other
than making them both seem immature and out of step with the significance
of what's going on elsewhere.
This is also a middle book of a trilogy, and it shows. It provides a
satisfying increase in intensity and gets the true plot of the trilogy
well underway, but nothing is resolved and a lot of new questions and plot
threads are raised. I had similar problems with Cold Fire, the middle book of the other Kate Elliott trilogy I've
read, and this book is 200 pages longer. Elliott loves world-building and
huge, complex plots; I have a soft spot for them too, but they mean the
story is full of stuff, and it's hard to maintain the same level of reader
interest across all the complications and viewpoints.
That said, I truly love the world-building. Elliott gives her world
historical layers, with multiple levels of lost technology, lost history,
and fallen empires, and backs it up with enough set pieces and fragments
of invented history that I was enthralled. There are at least five major
factions with different histories, cultures, and approaches to technology,
and although they all share a history, they interpret that history in
fascinatingly different ways. This world feels both lived in and full of
important mysteries.
Elliott also has a knack for backing the ambitions of her characters with
symbolism that defines the shape of that ambition. The title comes from a
(translated) verse of an in-universe song called the Hymn of Leaving,
which is sung at funerals and is about the flight on generation ships from
the now-lost Celestial Empire, the founding myth of this region of space:
Crossing the ocean of stars we leave our home behind us.
We are the spears cast at the furious heaven
And we will burn one by one into ashes
As with the last sparks we vanish.
This memory we carry to our own death which awaits us
And from which none of us will return.
Do not forget. Goodbye forever.
This is not great poetry, but it explains so much about the psychology of
the characters. Sun repeatedly describes herself and her allies as spears
cast at the furious heaven. Her mother's life mission was to make Chaonia
a respected independent power. Hers is much more than that, reaching back
into myth for stories of impossible leaps into space, burning brightly
against the hostile power of the universe itself.
A question about a series like this is why one should want to read about a
gender-swapped Alexander the Great in space, rather than just reading
about Alexander himself. One good (and sufficient) answer is that both
the gender swap and the space parts are inherently interesting. But the
other place that Elliott uses the science fiction background is to give
Sun motives beyond sheer personal ambition.
At a critical moment in the story, just like Alexander, Sun takes a detour
to consult an Oracle. Because this is a science fiction novel, it's a
great SF set piece involving a mysterious AI. But also because this is a
science fiction story, Sun doesn't only ask about her personal ambitions.
I won't spoil the exact questions; I think the moment is better not
knowing what she'll ask. But they're science fiction questions, reader
questions, the kinds of things Elliott has been building curiosity about
for a book and a half by the time we reach that scene. Half the fun of
reading a good epic space opera is learning the mysteries hidden in the
layers of world-building. Aligning the goals of the protagonist with the
goals of the reader is a simple storytelling trick, but oh, so effective.
Structurally, this is not that great of a book. There's a lot of build-up
and only some payoff, and there were several bits I found grating. But I
am thoroughly invested in this universe now. The third book can't come
soon enough.
Followed by Lady Chaos, which is still being written at the time of
this review.
Rating: 7 out of 10
In my recent post about data archiving to removable media, I laid out the difference between backing up and archiving, and also said I d evaluate git-annex and dar. This post evaluates git-annex. The next will look at dar, and then I ll make a comparison post.
What is git-annex?
git-annex is a fantastic and versatile program that does well, it s one of those things that can do so much that it s a bit hard to describe. Its homepage says:
git-annex allows managing large files with git, without storing the file contents in git. It can sync, backup, and archive your data, offline and online. Checksums and encryption keep your data safe and secure. Bring the power and distributed nature of git to bear on your large files with git-annex.
I think the particularly interesting features of git-annex aren t actually included in that list. Among the features of git-annex that make it shine for this purpose, its location tracking is key. git-annex can know exactly which device has which file at which version at all times. Combined with its preferred content settings, this lets you very easily say things like:
I want exactly 1 copy of every file to exist within the set #1 of backup drives. Here s a drive in that set; copy to it whatever needs to be copied to satisfy that requirement.
Now I have another set of backup drives. Periodically I will swap sets offsite. Copy whatever is needed to this drive in the second set, making sure that there is 1 copy of every file within this set as well, regardless of what s in the first set.
Here s a directory I want to use to track the status of everything else. I don t want any copies at all here.
git-annex can be set to allow a configurable amount of free space to remain on a device, and it will fill it up with whatever copies are necessary up until it hits that limit. Very convenient!
git-annex will store files in a folder structure that mirrors the origin folder structure, in plain files just as they were. This maximizes the ability for a future person to access the content, since it is all viewable without any special tool at all. Of course, for things like optical media, git-annex will essentially be creating what amounts to incrementals. To obtain a consistent copy of the original tree, you would still need to use git-annex to process (export) the archives.
git-annex challenges
In my prior post, I related some challenges with git-annex. The biggest of them quite poor performance of the directory special remote when dealing with many files has been resolved by Joey, git-annex s author! That dramatically improves the git-annex use scenario here! The fixing commit is in the source tree but not yet in a release.
git-annex no doubt may still have performance challenges with repositories in the 100,000+-range, but in that order of magnitude it now looks usable. I m not sure about 1,000,000-file repositories (I haven t tested); there is a page about scalability.
A few other more minor challenges remain:
git-annex doesn t really preserve POSIX attributes; for instance, permissions, symlink destinations, and timestamps are all not preserved. Of these, timestamps are the most important for my particular use case.
If your data set to archive contains Git repositories itself, these will not be included.
I worked around the timestamp issue by using the mtree-netbsd package in Debian. mtree writes out a summary of files and metadata in a tree, and can restore them. To save:
mtree -c -R nlink,uid,gid,mode -p /PATH/TO/REPO -X <(echo './.git') > /tmp/spec
And, after restoration, the timestamps can be applied with:
mtree -t -U -e < /tmp/spec
Walkthrough: initial setup
To use git-annex in this way, we have to do some setup. My general approach is this:
There is a source of data that lives outside git-annex. I'll call this $SOURCEDIR.
I'm going to name the directories holding my data $REPONAME.
There will be a "coordination" git-annex repo. It will hold metadata only, and no data. This will let us track where things live. I'll call it $METAREPO.
There will be drives. For this example, I'll call their mountpoints $DRIVE01 and $DRIVE02. For easy demonstration purposes, I used a ZFS dataset with a refquota set (to observe the size handling), but I could have as easily used a LVM volume, btrfs dataset, loopback filesystem, or USB drive. For optical discs, this would be a staging area or a UDF filesystem.
Let's get started! I've set all these shell variables appropriately for this example, and REPONAME to "testdata". We'll begin by setting up the metadata-only tracking repo.
$ REPONAME=testdata
$ mkdir "$METAREPO"
$ cd "$METAREPO"
$ git init
$ git config annex.thin true
There is a sort of complicated topic of how git-annex stores files in a repo, which varies depending on whether the data for the file is present in a given repo, and whether the file is locked or unlocked. Basically, the options I use here cause git-annex to mostly use hard links instead of symlinks or pointer files, for maximum compatibility with non-POSIX filesystems such as NTFS and UDF, which might be used on these devices. thin is part of that.
Let's continue:
$ git annex init 'local hub'
init local hub ok
(recording state in git...)
$ git annex wanted . "include=* and exclude=$REPONAME/*"
wanted . ok
(recording state in git...)
In a bit, we are going to import the source data under the directory named $REPONAME (here, testdata). The wanted command says: in this repository (represented by the bare dot), the files we want are matched by the rule that says eveyrthing except what's under $REPONAME. In other words, we don't want to make an unnecessary copy here.
Because I expect to use an mtree file as documented above, and it is not under $REPONAME/, it will be included. Let's just add it and tweak some things.
$ touch mtree
$ git annex add mtree
add mtree
ok
(recording state in git...)
$ git annex sync
git-annex sync will change default behavior to operate on --content in a future version of git-annex. Recommend you explicitly use --no-content (or -g) to prepare for that change. (Or you can configure annex.synccontent)
commit
[main (root-commit) 6044742] git-annex in local hub
1 file changed, 1 insertion(+)
create mode 120000 mtree
ok
$ ls -l
total 9
lrwxrwxrwx 1 jgoerzen jgoerzen 178 Jun 15 22:31 mtree -> .git/annex/objects/pX/ZJ/...
OK! We've added a file, and it got transformed into a symlink. That's the thing I said we were going to avoid, so:
git annex adjust --unlock-present
adjust
Switched to branch 'adjusted/main(unlockpresent)'
ok
$ ls -l
total 1
-rw-r--r-- 2 jgoerzen jgoerzen 0 Jun 15 22:31 mtree
You'll notice it transformed into a hard link (nlinks=2) file. Great! Now let's import the source data. For that, we'll use the directory special remote.
$ git annex initremote source type=directory directory=$SOURCEDIR importtree=yes \
encryption=none
initremote source ok
(recording state in git...)
$ git annex enableremote source directory=$SOURCEDIR
enableremote source ok
(recording state in git...)
$ git config remote.source.annex-readonly true
$ git config annex.securehashesonly true
$ git config annex.genmetadata true
$ git config annex.diskreserve 100M
$ git config remote.source.annex-tracking-branch main:$REPONAME
OK, so here we created a new remote named "source". We enabled it, and set some configuration. Most notably, that last line causes files from "source" to be imported under $REPONAME/ as we wanted earlier. Now we're ready to scan the source.
$ git annex sync
At this point, you'll see git-annex computing a hash for every file in the source directory.
I can verify with du that my metadata-only repo only uses 14MB of disk space, while my source is around 4GB.
Now we can see what git-annex thinks about file locations:
$ git-annex whereis less
whereis mtree (1 copy)
8aed01c5-da30-46c0-8357-1e8a94f67ed6 -- local hub [here]
ok
whereis testdata/[redacted] (0 copies)
The following untrusted locations may also have copies:
9e48387e-b096-400a-8555-a3caf5b70a64 -- [source]
failed
... many more lines ...
So remember we said we wanted mtree, but nothing under testdata, under this repo? That's exactly what we got. git-annex knows that the files under testdata can be found under the "source" special remote, but aren't in any git-annex repo -- yet. Now we'll start adding them.
Walkthrough: removable drives
I've set up two 500MB filesystems to represent removable drives. We'll see how git-annex works with them.
$ cd $DRIVE01
$ df -h .
Filesystem Size Used Avail Use% Mounted on
acrypt/no-backup/annexdrive01 500M 1.0M 499M 1% /acrypt/no-backup/annexdrive01
$ git clone $METAREPO
Cloning into 'testdata'...
done.
$ cd $REPONAME
$ git config annex.thin true
$ git annex init "test drive #1"
$ git annex adjust --hide-missing --unlock
adjust
Switched to branch 'adjusted/main(hidemissing-unlocked)'
ok
$ git annex sync
OK, that's the initial setup. Now let's enable the source remote and configure it the same way we did before:
$ git annex enableremote source directory=$SOURCEDIR
enableremote source ok
(recording state in git...)
$ git config remote.source.annex-readonly true
$ git config remote.source.annex-tracking-branch main:$REPONAME
$ git config annex.securehashesonly true
$ git config annex.genmetadata true
$ git config annex.diskreserve 100M
Now, we'll add the drive to a group called "driveset01" and configure what we want on it:
$ git annex group . driveset01
$ git annex wanted . '(not copies=driveset01:1)'
What this does is say: first of all, this drive is in a group named driveset01. Then, this drive wants any files for which there isn't already at least one copy in driveset01.
Now let's load up some files!
$ git annex sync --content
As the messages fly by from here, you'll see it mentioning that it got mtree, and then various files from "source" -- until, that is, the filesystem had less than 100MB free, at which point it complained of no space for the rest. Exactly like we wanted!
Now, we need to teach $METAREPO about $DRIVE01.
$ cd $METAREPO
$ git remote add drive01 $DRIVE01/$REPONAME
$ git annex sync drive01
git-annex sync will change default behavior to operate on --content in a future version of git-annex. Recommend you explicitly use --no-content (or -g) to prepare for that change. (Or you can configure annex.synccontent)
commit
On branch adjusted/main(unlockpresent)
nothing to commit, working tree clean
ok
merge synced/main (Merging into main...)
Updating d1d9e53..817befc
Fast-forward
(Merging into adjusted branch...)
Updating 7ccc20b..861aa60
Fast-forward
ok
pull drive01
remote: Enumerating objects: 214, done.
remote: Counting objects: 100% (214/214), done.
remote: Compressing objects: 100% (95/95), done.
remote: Total 110 (delta 6), reused 0 (delta 0), pack-reused 0
Receiving objects: 100% (110/110), 13.01 KiB 1.44 MiB/s, done.
Resolving deltas: 100% (6/6), completed with 6 local objects.
From /acrypt/no-backup/annexdrive01/testdata
* [new branch] adjusted/main(hidemissing-unlocked) -> drive01/adjusted/main(hidemissing-unlocked)
* [new branch] adjusted/main(unlockpresent) -> drive01/adjusted/main(unlockpresent)
* [new branch] git-annex -> drive01/git-annex
* [new branch] main -> drive01/main
* [new branch] synced/main -> drive01/synced/main
ok
OK! This step is important, because drive01 and drive02 (which we'll set up shortly) won't necessarily be able to reach each other directly, due to not being plugged in simultaneously. Our $METAREPO, however, will know all about where every file is, so that the "wanted" settings can be correctly resolved. Let's see what things look like now:
$ git annex whereis less
whereis mtree (2 copies)
8aed01c5-da30-46c0-8357-1e8a94f67ed6 -- local hub [here]
b46fc85c-c68e-4093-a66e-19dc99a7d5e7 -- test drive #1 [drive01]
ok
whereis testdata/[redacted] (1 copy)
b46fc85c-c68e-4093-a66e-19dc99a7d5e7 -- test drive #1 [drive01] The following untrusted locations may also have copies:
9e48387e-b096-400a-8555-a3caf5b70a64 -- [source]
ok
If I scroll down a bit, I'll see the files past the 400MB mark that didn't make it onto drive01. Let's add another example drive!
Walkthrough: Adding a second drive
The steps for $DRIVE02 are the same as we did before, just with drive02 instead of drive01, so I'll omit listing it all a second time. Now look at this excerpt from whereis:
whereis testdata/[redacted] (1 copy)
b46fc85c-c68e-4093-a66e-19dc99a7d5e7 -- test drive #1 [drive01]
The following untrusted locations may also have copies:
9e48387e-b096-400a-8555-a3caf5b70a64 -- [source]
ok
whereis testdata/[redacted] (1 copy)
c4540343-e3b5-4148-af46-3f612adda506 -- test drive #2 [drive02]
The following untrusted locations may also have copies:
9e48387e-b096-400a-8555-a3caf5b70a64 -- [source]
ok
Look at that! Some files on drive01, some on drive02, some neither place. Perfect!
Walkthrough: Updates
So I've made some changes in the source directory: moved a file, added another, and deleted one. All of these were copied to drive01 above. How do we handle this?
First, we update the metadata repo:
$ cd $METAREPO
$ git annex sync
$ git annex dropunused all
OK, this has scanned $SOURCEDIR and noted changes. Let's see what whereis says:
$ git annex whereis less
...
whereis testdata/cp (0 copies)
The following untrusted locations may also have copies:
9e48387e-b096-400a-8555-a3caf5b70a64 -- [source]
failed
whereis testdata/file01-unchanged (1 copy)
b46fc85c-c68e-4093-a66e-19dc99a7d5e7 -- test drive #1 [drive01] The following untrusted locations may also have copies:
9e48387e-b096-400a-8555-a3caf5b70a64 -- [source]
ok
So this looks right. The file I added was a copy of /bin/cp. I moved another file to one named file01-unchanged. Notice that it realized this was a rename and that the data still exists on drive01.
Well, let's update drive01.
$ cd $DRIVE01/$REPONAME
$ git annex sync --content
Looking at the testdata/ directory now, I see that file01-unchanged has been renamed, the deleted file is gone, but cp isn't yet here -- probably due to space issues; as it's new, it's undefined whether it or some other file would fill up free space. Let's work along a few more commands.
$ git annex get --auto
$ git annex drop --auto
$ git annex dropunused all
And now, let's make sure metarepo is updated with its state.
$ cd $METAREPO
$ git annex sync
We could do the same for drive02. This is how we would proceed with every update.
Walkthrough: Restoration
Now, we have bare files at reasonable locations in drive01 and drive02. But, to generate a consistent restore, we need to be able to actually do an export. Otherwise, we may have files with old names, duplicate files, etc. Let's assume that we lost our source and metadata repos and have to restore from scratch. We'll make a new $RESTOREDIR. We'll begin with drive01 since we used it most recently.
$ mv $METAREPO $METAREPO.disabled
$ mv $SOURCEDIR $SOURCEDIR.disabled
$ git clone $DRIVE01/$REPONAME $RESTOREDIR
$ cd $RESTOREDIR
$ git config annex.thin true
$ git annex init "restore"
$ git annex adjust --hide-missing --unlock
Now, we need to connect the drive01 and pull the files from it.
$ git remote add drive01 $DRIVE01/$REPONAME
$ git annex sync --content
Now, repeat with drive02:
$ git remote add drive02 $DRIVE02/$REPONAME
$ git annex sync --content
Now we've got all our content back! Here's what whereis looks like:
whereis testdata/file01-unchanged (3 copies)
3d663d0f-1a69-4943-8eb1-f4fe22dc4349 -- restore [here]
9e48387e-b096-400a-8555-a3caf5b70a64 -- source
b46fc85c-c68e-4093-a66e-19dc99a7d5e7 -- test drive #1 [origin]
ok
...
I was a little surprised that drive01 didn't seem to know what was on drive02. Perhaps that could have been remedied by adding more remotes there? I'm not entirely sure; I'd thought would have been able to do that automatically.
Conclusions
I think I have demonstrated two things:
First, git-annex is indeed an extremely powerful tool. I have only scratched the surface here. The location tracking is a neat feature, and being able to just access the data as plain files if all else fails is nice for future users.
Secondly, it is also a complex tool and difficult to get right for this purpose (I think much easier for some other purposes). For someone that doesn't live and breathe git-annex, it can be hard to get right. In fact, I'm not entirely sure I got it right here. Why didn't drive02 know what files were on drive01 and vice-versa? I don't know, and that reflects some kind of misunderstanding on my part about how metadata is synced; perhaps more care needs to be taken in restore, or done in a different order, than I proposed. I initially tried to do a restore by using git annex export to a directory special remote with exporttree=yes, but I couldn't ever get it to actually do anything, and I don't know why.
These two cut against each other. On the one hand, the raw accessibility of the data to someone with no computer skills is unmatched. On the other hand, I'm not certain I have the skill to always prepare the discs properly, or to do a proper consistent restore.
Just a few days back we came to know about the horrific Train Crash that happened in Odisha (Orissa). There are some things that are known and somethings that can be inferred by observance. Sadly, it seems the incident is going to be covered up . Some of the facts that have not been contested in the public domain are that there were three lines. One loop line on which the Goods Train was standing and there was an up and a down line. So three lines were there. Apparently, the signalling system and the inter-locking system had issues as highlighted by an official about a month back. That letter, thankfully is in the public domain and I have downloaded it as well. It s a letter that goes to 4 pages. The RW is incensed that the letter got leaked and is in public domain. They are blaming everyone and espousing conspiracy theories rather than taking the minister to task. Incidentally, the Minister has three ministries that he currently holds. Ministry of Communication, Ministry of Electronics and Information Technology (MEIT), and Railways Ministry. Each Ministry in itself is important and has revenues of more than 6 lakh crore rupees. How he is able to do justice to all the three ministries is beyond me
The other thing is funds both for safety and relaying of tracks has been either not sanctioned or unutilized. In fact, CAG and the Railway Brass had shared how derailments have increased and unfulfilled vacancies but they were given no importance In fact, not talking about safety in the recently held Chintan Shivir (brainstorming session) tells you how much the Govt. is serious about safety. In fact, most of the programme was on high speed rail which is a white elephant. I have shared a whitepaper done by RW in the U.S. that tells how high-speed rail doesn t make economic sense. And that is an economy that is 20 times + the Indian Economy. Even the Chinese are stopping with HSR as it doesn t make economic sense.
Incidentally, Air Fares again went up 200% yesterday. Somebody shared in the region of 20k + for an Air ticket from their place to Bangalore
Coming back to the story itself. the Goods Train was on the loopline. Some say it was a little bit on the outer, some say otherwise, but it is established that it was on the loopline. This is standard behavior on and around Railway Stations around the world. Whether it was in the Inner or Outer doesn t make much of a difference with what happened next. The first train that collided with the goods train was the 12864 (SMVB-HWH) Yashwantpur Howrah Express and got derailed on to the next track where from the opposite direction 12841 (Shalimar- Bangalore) Coramandel Express was coming. Now they have said that around 300 people have died and that seems to be part of the cover-up. Both the trains are long trains, having between 23 odd coaches each. Even if you have reserved tickets you have 80 odd people in a coach and usually in most of these trains, it is at least double of that. Lot of money goes to TC and then above (Corruption). The Railway fares have gone up enormously but that s a question for perhaps another time . So at the very least, we could be looking at more than 1000 people having died. The numbers are being under-reported so that nobody has to take responsibility. The Railways itself has told that it is unable to identify 80% of the people who have died. This means that 80% were unreserved ticket holders or a majority of them. There have been disturbing images as how bodies have been flung over on tractors and whatnot to be either buried or cremated without a thought. We are in peak summer season so bodies will start to rot within 24-48 hours No arrangements made to cool the bodies and take some information and identifying marks or whatever. The whole thing being done in a very callous manner, not giving dignity to even those who have died for no fault of their own. The dissent note also tells that a cover-up is also in the picture. Apparently, India doesn t have nor does it feel to have a need for something like the NTSB that the U.S. used when it hauled both the plane manufacturer (Boeing) and the FAA when the 737 Max went down due to improper data collection and sharing of data with pilots. And with no accountability being fixed to Minister or any of the senior staff, a small junior staff person may be fired. Perhaps the same official that actually told them about the signal failures almost 3 months back
There were and are also some reports that some jugaadu /temporary fixes were applied to signalling and inter-locking just before this incident happened. I do not know nor confirm one way or the other if the above happened. I can however point out that if such a thing happened, then usually a traffic block is announced and all traffic on those lines are stopped. This has been the thing I know for decades. Traveling between Mumbai and Pune multiple times over the years am aware about traffic block. If some repair work was going on and it wasn t able to complete the work within the time-frame then that may well have contributed to the accident. There is also a bit muddying of the waters where it is being said that one of the trains was 4 hours late, which one is conflicting stories.
On top of the whole thing, they have put the case to be investigated by CBI and hinting at sabotage. They also tried to paint a religious structure as mosque, later turned out to be a temple. The RW says done by Muslims as it was Friday not taking into account as shared before that most Railway maintenance works are usually done between Friday Monday. This is a practice followed not just in India but world over.
There has been also move over a decade to remove wooden sleepers and have concrete sleepers. Unlike the wooden ones they do not expand and contract as much and their life is much more longer than the wooden ones. Funds had been marked (although lower than last few years) but not yet spent. As we know in case of any accident, it is when all the holes in cheese line up it happens. Fukushima is a great example of that, no sea wall even though Japan is no stranger to Tsunamis. External power at the same level as the plant. (10 meters above sea-level), no training for cascading failures scenarios which is what happened. The Days mini-series shares some but not all the faults that happened at Fukushima and the Govt. response to it. There is a difference though, the Japanese Prime Minister resigned on moral grounds. Here, nor the PM, nor the Minister would be resigning on moral grounds or otherwise :(. Zero accountability and that was partly a natural disaster, here it s man-made. In fact, both the Minister and the Prime Minister arrived with their entourages, did a PR blitzkrieg showing how concerned they are. Within 50 hours, the lines were cleared. The part-time Railway Minister shared that he knows the root cause and then few hours later has given the case to CBI. All are saying, wait for the inquiry report. To date, none of the accidents even in this Govt. has produced an investigation report. And even if it did, I am sure it will whitewash as it did in case of Adani as I had shared before in the previous blog post. Incidentally, it is reported that Adani paid off some of its debt, but when questioned as to where they got the money, complete silence on that part :(. As can be seen cover-up after cover-up
FWIW, the Coramandel Express is known as the Migrant train so has a huge number of passengers, the other one which was collided with is known as sick train as huge number of cancer patients use it to travel to Chennai and come back
Demonetization 2.0
Few days back, India announced demonetization 2.0. Surprised, don t be. Apparently, INR 2k/- is being used for corruption and Mr. Modi is unhappy about it. He actually didn t like the INR 2k/- note but was told that it was needed, who told him we are unaware to date. At that time the RBI Governor was Mr. Urjit Patel who didn t say about INR 2k/- he had said that INR 1k/- note redesigned would come in the market. That has yet to happen. What has happened is that just like INR 500/- and INR 1k/- note is concerned, RBI will no longer honor the INR 2k/- note. Obviously, this has made our neighbors angry, namely Nepal, Sri Lanka, Bhutan etc. who do some trading with us. 2 Deccan heraldcolumns share the limelight on it. Apparently, India wants to be the world s currency reserve but doesn t want to play by the rules for everyone else. It was pointed out that both the U.S. and Singapore had retired their currencies but they will honor that promise even today. The Singapore example being a bit closer (as it s in Asia) is perhaps a bit more relevant than the U.S. one. Singapore retired the SGD $10,000 as of 2014 but even in 2022, it remains as legal tender. They also retired the SGD $1,000 in 2020 but still remains legal tender.
So let s have a fictitious example to illustrate what is meant by what Singapore has done. Let s say I go to Singapore, rent a flat, and find a $1000 note in that house somewhere. Both practically and theoretically, I could go down to any of the banks, get the amount transferred to my wallet, bank account etc. and nobody will question. Because they have promised the same. Interestingly, the Singapore Dollar has been pretty resilient against the USD for quite a number of years vis-a-vis other Asian currencies.
Most of the INR 2k/- notes were also found and exchanged in Gujarat in just a few days (The PM and HM s state.). I am sure you are looking into the mental gymnastics that the RW indulge in :(. What is sadder that most of the people who try to defend can t make sense one way or the other and start to name-call and get personal as they have nothing else
Disability questions dropped in NHFS-6
Just came to know today that in the upcoming National Family Health Survey-6 disability questions are being dropped. Why is this important. To put it simply, if you don t have numbers, you won t and can t make policies for them. India is one of the worst countries to live if you are disabled. The easiest way to share to draw attention is most Railway platforms are not at level with people. Just as Mick Lynch shares in the UK, the same is pretty much true for India too. Meanwhile in Europe, they do make an effort to be level so even disabled people have some dignity. If your public transport is sorted, then people would want much more and you will be obligated to provide for them as they are citizens. Here, we have had many reports of women being sexually molested when being transferred from platform to coach irrespective of their age or whatnot The main takeaway is if you do not have their voice, you won t make policies for them. They won t go away but you will make life hell for them. One thing to keep in mind that most people assume that most people are disabled from birth. This may or may not be true. For e.g. in the above triple Railways accidents, there are bound to be disabled people or newly disabled people who were healthy before the accident. The most common accident is road accidents, some involving pedestrians and vehicles or both, the easiest is Ministry of Road Transport data that says 4,00,000 people sustained injuries in 2021 alone in road mishaps. And this is in a country where even accidents are highly under-reported, for more than one reason. The biggest reason especially in 2 and 4 wheeler is the increased premium they would have to pay if in an accident, so they usually compromise with the other and pay off the Traffic Inspector.
Sadly, I haven t read a new book, although there are a few books I m looking forward to have. People living in India and neighbors please be careful as more heat waves are expected. Till later.
QNAP TS-453mini product photo
The logo for QNAP HappyGet 2 and Blizzard s StarCraft 2 side by side
Thermalright AXP120-X67, AMD Ryzen 5 PRO 5650GE, ASRock Rack X570D4I-2T, all assembled and running on a flat surface
Memtest86 showing test progress, taken from IPMI remote control window
Screenshot of PCIe 16x slot bifurcation options in UEFI settings, taken from IPMI remote control window
Internal image of Silverstone CS280 NAS build. Image stolen from ServeTheHome
Internal image of Silverstone CS280 NAS build. Image stolen from ServeTheHome
NAS build in Silverstone SUGO 14, mid build, panels removed
Silverstone SUGO 14 from the front, with hot swap bay installed
Storage SSD loaded into hot swap sled
TrueNAS Dashboard screenshot in browser window
The final system, powered up
~
PS: Debian uploads for bubble-gum are moving along. Please if you have any packaging you need done in Debian proper, let me know and I will get on it, time allowing of course.
There is a bit of context that needs to be shared before I get to this and would be a long one. For reasons known and unknown, I have a lot of sudden electricity outages. Not just me, all those who are on my line. A discussion with a lineman revealed that around 200+ families and businesses are on the same line and when for whatever reason the electricity goes for all. Even some of the traffic lights don t work. This affects software more than hardware or in some cases, both. And more specifically HDD s are vulnerable. I had bought an APC unit several years for precisely this, but over period of time it just couldn t function and trips also when the electricity goes out. It s been 6-7 years so can t even ask customer service to fix the issue and from whatever discussions I have had with APC personnel, the only meaningful difference is to buy a new unit but even then not sure this is an issue that can be resolved, even with that.
That comes to the issue that happens once in a while where the system fsck is unable to repair /home and you need to use an external pen drive for the same. This is my how my hdd stacks up 
The main argument as have shared before is that Indian Govt. thinks we need our home grown CPU and while I have no issues with that, as shared before except for RISC-V there is no other space where India could look into doing that. Especially after the Chip Act, Biden has made that any new fabs or any new thing in chip fabrication will only be shared with 
. Backups certainly make a lot of sense, especially 
Edited 2023-08-25 01:32 BST to correct a slip.
Welcome to the 43th post in the 
Turns out maybe not so much (yet ?). As the 
Now, this is of course entirely possibly that not all possible venues
for speedups were exploited in how the action setup was setup. If so,
please file an issue at the 
Uhm, salsa is not resolving:
Since some time I ve been thinking about making myself a top or a dress with a wide gathered neckline that can be work at different widths, including off-the-shoulders.
A few years ago I ve been gifted a cut of nice, thin white fabric with a print of lines and lozenges that isn t uniform along the fabric, but looks like it was designed for some specific garment, and it was waiting in my stash for a suitable pattern.
And a few days ago, during a Sunday lunch, there was an off-hand mention of a dress from the late 1970s which had an elastic in the neckline, so that it could be optionally worn off-the-shoulders.
And something snapped in place.
I had plans for that afternoon, but they were scrapped, and I started to draw, measure, cut rectangles of fabric, pin and measure again, cut more fabric.
The part of the fabric with large vertical stripes had two different widths: I could have made the back narrower, but I decided to just keep a strip with narrower lines to one side.
The fabric also didn t have a full second strip of lozenges, so I had to hem it halfway through it.
The casing for the elastic was pieced from various scraps, but at least I was able to match the lines on the center front and back, even if they are different. Not that it matters a lot, since it s all hidden in the gathering, but I would have known.
And since I was working on something definitely modern, even if made out of squares and rectangles, of course I decided to hand-sew everything, mostly to be able to use quite small sewing allowances, since the fabric was pretty thin.
In my stash I had a piece of swimsuit elastic that feels nice, looks nice and makes a knot that doesn t slip, so I used it. It s a perfect match, except for the neon yellow colour, which I do like, but maybe is a bit too high visibility? I will see if the haberdasher has the same elastic in dark blue, but right now this will do.
It was a quick project anyway: by the end of the working week the top was finished; I think that on a sewing machine it would be easy to make it in a day.
And it can be worn off the shoulders! Which is something I will probably never do in public (and definitely not outdoors), but now if I wanted I could! :D
As usual, the pattern (for what pattern there is) and instructions are 
KDE Akademy 2023
KDE Konsole snap
The DebConf video team has been
I m six months into my journey of building a business which means its time to reflect and review the goals I set for the year.